{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "11_Convolutional_Neural_Networks",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "metadata": {
        "id": "bOChJSNXtC9g",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Convolutional Neural Networks"
      ]
    },
    {
      "metadata": {
        "id": "OLIxEDq6VhvZ",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/logo.png\" width=150>\n",
        "\n",
        "In this lesson we will learn the basics of Convolutional Neural Networks (CNNs) applied to text for natural language processing (NLP) tasks. CNNs are traditionally used on images and there are plenty of [tutorials](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html) that cover this. But we're going to focus on using CNN on text data which yields amazing results. \n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "VoMq0eFRvugb",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Overview\n",
        "\n",
        "The diagram below from this [paper](https://arxiv.org/abs/1510.03820) shows how 1D convolution is applied to the words in a sentence. "
      ]
    },
    {
      "metadata": {
        "id": "ziGJNhiQeiGN",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/cnn_text.png\" width=500>"
      ]
    },
    {
      "metadata": {
        "id": "qWro5T5qTJJL",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "* **Objective:**  Detect spatial substructure from input data.\n",
        "* **Advantages:** \n",
        "  * Small number of weights (shared)\n",
        "  * Parallelizable\n",
        "  * Detects spatial substrcutures (feature extractors)\n",
        "  * Interpretable via filters\n",
        "  * Used for in images/text/time-series etc.\n",
        "* **Disadvantages:**\n",
        "  * Many hyperparameters (kernel size, strides, etc.)\n",
        "  * Inputs have to be of same width (image dimensions, text length, etc.)\n",
        "* **Miscellaneous:** \n",
        "  * Lot's of deep CNN architectures constantly updated for SOTA performance"
      ]
    },
    {
      "metadata": {
        "id": "8nCsZGyWhI9f",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Filters"
      ]
    },
    {
      "metadata": {
        "id": "lxpgRzIjiVHv",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "At the core of CNNs are filters (weights, kernels, etc.) which convolve (slide) across our input to extract relevant features. The filters are initialized randomly but learn to pick up meaningful features from the input that aid in optimizing for the objective. We're going to teach CNNs in an unorthodox method where we entirely focus on applying it to 2D text data. Each input is composed of words and we will be representing each word as one-hot encoded vector which gives us our 2D input. The intuition here is that each filter represents a feature and we will use this filter on other inputs to capture the same feature. This is known as parameter sharing.\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/conv.gif\" width=400>"
      ]
    },
    {
      "metadata": {
        "id": "1kTABJyYj91S",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Loading PyTorch library\n",
        "!pip3 install torch"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "kz9D2rrdmSl9",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch\n",
        "import torch.nn as nn"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "1q1FiiIHXjI_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Our inputs are a batch of 2D text data. Let's make an input with 64 samples, where each sample has 8 words and each word is represented by a array of 10 values (one hot encoded with vocab size of 10). This gives our inputs the size (64, 8, 10). The [PyTorch CNN modules](https://pytorch.org/docs/stable/nn.html#convolution-functions) prefer inputs to have the channel dim (one hot vector dim in our case) to be in the second position, so our inputs are of shape (64, 10, 8)."
      ]
    },
    {
      "metadata": {
        "id": "tFfYwCcjZj79",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/cnn_text1.png\" width=400>"
      ]
    },
    {
      "metadata": {
        "id": "b6G2nBvOxR-e",
        "colab_type": "code",
        "outputId": "3b422df4-d2c6-4f2f-cb71-925453c54948",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Assume all our inputs have the same # of words\n",
        "batch_size = 64\n",
        "sequence_size = 8 # words per input\n",
        "one_hot_size = 10 # vocab size (num_input_channels)\n",
        "x = torch.randn(batch_size, one_hot_size, sequence_size)\n",
        "print(\"Size: {}\".format(x.shape))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([64, 10, 8])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "GJmtay_UZohM",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We want to convolve on this input using filters. For simplicity we will use just 5 filters that is of size (1, 2) and has the same depth as the number of channels (one_hot_size). This gives our filter a shape of (5, 2, 10) but recall that PyTorch CNN modules prefer to have the channel dim (one hot vector dim in our case) to be in the second position so the filter is of shape (5, 10, 2)."
      ]
    },
    {
      "metadata": {
        "id": "ZJF0l88qb-21",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/cnn_text2.png\" width=400>"
      ]
    },
    {
      "metadata": {
        "id": "WMK2TzgDxR8B",
        "colab_type": "code",
        "outputId": "59562322-13b9-43cf-ec5f-04d692566364",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "# Create filters for a conv layer\n",
        "out_channels = 5 # of filters\n",
        "kernel_size = 2 # filters size 2\n",
        "conv1 = nn.Conv1d(in_channels=one_hot_size, out_channels=out_channels, kernel_size=kernel_size)\n",
        "print(\"Size: {}\".format(conv1.weight.shape))\n",
        "print(\"Filter size: {}\".format(conv1.kernel_size[0]))\n",
        "print(\"Padding: {}\".format(conv1.padding[0]))\n",
        "print(\"Stride: {}\".format(conv1.stride[0]))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([5, 10, 2])\n",
            "Filter size: 2\n",
            "Padding: 0\n",
            "Stride: 1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "lAcYxhDIbeWE",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "When we apply this filter on our inputs, we receive an output of shape (64, 5, 7). We get 64 for the batch size, 5 for the channel dim because we used 5 filters and 7 for the conv outputs because:\n",
        "\n",
        "$\\frac{W - F + 2P}{S} + 1 = \\frac{8 - 2 + 2(0)}{1} + 1 = 7$\n",
        "\n",
        "where:\n",
        "  * W: width of each input\n",
        "  * F: filter size\n",
        "  * P: padding\n",
        "  * S: stride"
      ]
    },
    {
      "metadata": {
        "id": "2c_KKtP4hrJx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/cnn_text3.png\" width=400>"
      ]
    },
    {
      "metadata": {
        "id": "yjxtrM89xR5a",
        "colab_type": "code",
        "outputId": "5892ff1d-595f-4528-b84a-dcd4295181f3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Convolve using filters\n",
        "conv_output = conv1(x)\n",
        "print(\"Size: {}\".format(conv_output.shape))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([64, 5, 7])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "vwTtF7bBuZvF",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Pooling"
      ]
    },
    {
      "metadata": {
        "id": "VXBbKPs1ua9G",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "The result of convolving filters on an input is a feature map. Due to the nature of convolution and overlaps, our feature map will have lots of redundant information. Pooling is a way to summarize a high-dimensional feature map into a lower dimensional one for simplified downstream computation. The pooling operation can be the max value, average, etc. in a certain receptive field.\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/pool.jpeg\" width=450>"
      ]
    },
    {
      "metadata": {
        "id": "VCag6lk2mSwU",
        "colab_type": "code",
        "outputId": "27fd0640-c1c5-467b-a77e-c19eb50e6749",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Max pooling\n",
        "kernel_size = 2\n",
        "pool1 = nn.MaxPool1d(kernel_size=kernel_size, stride=2, padding=0)\n",
        "pool_output = pool1(conv_output)\n",
        "print(\"Size: {}\".format(pool_output.shape))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Size: torch.Size([64, 5, 3])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "c_e4QRFwvTt8",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "$\\frac{W-F}{S} + 1 = \\frac{7-2}{2} + 1 =  \\text{floor }(2.5) + 1 = 3$"
      ]
    },
    {
      "metadata": {
        "id": "l9rL1EWIfi-y",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# CNNs on text"
      ]
    },
    {
      "metadata": {
        "id": "aWtHDOJgHZvk",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "We're going use convolutional neural networks on text data which typically involves convolving on the character level representation of the text to capture meaningful n-grams. \n",
        "\n",
        "You can easily use this set up for [time series](https://arxiv.org/abs/1807.10707) data or [combine it](https://arxiv.org/abs/1808.04928) with other networks. For text data, we will create filters of varying kernel sizes (1,2), (1,3), and (1,4) which act as feature selectors of varying n-gram sizes. The outputs are concated and fed into a fully-connected layer for class predictions. In our example, we will be applying 1D convolutions on letter in a word. In the [embeddings notebook](https://colab.research.google.com/github/GokuMohandas/practicalAI/blob/master/notebooks/12_Embeddings.ipynb), we will apply 1D convolutions on words in a sentence.\n",
        "\n",
        "**Word embeddings**: capture the temporal correlations among\n",
        "adjacent tokens so that similar words have similar representations. Ex. \"New Jersey\" is close to \"NJ\" is close to \"Garden State\", etc.\n",
        "\n",
        "**Char embeddings**: create representations that map words at a character level. Ex. \"toy\" and \"toys\" will be close to each other."
      ]
    },
    {
      "metadata": {
        "id": "bVBZxbaAtS9u",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Set up"
      ]
    },
    {
      "metadata": {
        "id": "y8QSdEcDtXUs",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import os\n",
        "from argparse import Namespace\n",
        "import collections\n",
        "import copy\n",
        "import json\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import re\n",
        "import torch"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "VADCXjMwtXYN",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Set Numpy and PyTorch seeds\n",
        "def set_seeds(seed, cuda):\n",
        "    np.random.seed(seed)\n",
        "    torch.manual_seed(seed)\n",
        "    if cuda:\n",
        "        torch.cuda.manual_seed_all(seed)\n",
        "        \n",
        "# Creating directories\n",
        "def create_dirs(dirpath):\n",
        "    if not os.path.exists(dirpath):\n",
        "        os.makedirs(dirpath)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "mpiCYECstXbT",
        "colab_type": "code",
        "outputId": "d836f55c-3fce-4b3e-ea6b-bc32c5fb5b10",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "cell_type": "code",
      "source": [
        "# Arguments\n",
        "args = Namespace(\n",
        "    seed=1234,\n",
        "    cuda=False,\n",
        "    shuffle=True,\n",
        "    data_file=\"names.csv\",\n",
        "    split_data_file=\"split_names.csv\",\n",
        "    vectorizer_file=\"vectorizer.json\",\n",
        "    model_state_file=\"model.pth\",\n",
        "    save_dir=\"names\",\n",
        "    train_size=0.7,\n",
        "    val_size=0.15,\n",
        "    test_size=0.15,\n",
        "    num_epochs=20,\n",
        "    early_stopping_criteria=5,\n",
        "    learning_rate=1e-3,\n",
        "    batch_size=64,\n",
        "    num_filters=100,\n",
        "    dropout_p=0.1,\n",
        ")\n",
        "\n",
        "# Set seeds\n",
        "set_seeds(seed=args.seed, cuda=args.cuda)\n",
        "\n",
        "# Create save dir\n",
        "create_dirs(args.save_dir)\n",
        "\n",
        "# Expand filepaths\n",
        "args.vectorizer_file = os.path.join(args.save_dir, args.vectorizer_file)\n",
        "args.model_state_file = os.path.join(args.save_dir, args.model_state_file)\n",
        "\n",
        "# Check CUDA\n",
        "if not torch.cuda.is_available():\n",
        "    args.cuda = False\n",
        "args.device = torch.device(\"cuda\" if args.cuda else \"cpu\")\n",
        "print(\"Using CUDA: {}\".format(args.cuda))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Using CUDA: False\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "ptb4hJ4Bw8YU",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Data"
      ]
    },
    {
      "metadata": {
        "id": "bNxZQUqfmS0B",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import re\n",
        "import urllib"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "MBdQpUTQtMgu",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Upload data from GitHub to notebook's local drive\n",
        "url = \"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/data/surnames.csv\"\n",
        "response = urllib.request.urlopen(url)\n",
        "html = response.read()\n",
        "with open(args.data_file, 'wb') as fp:\n",
        "    fp.write(html)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "6PYCeGrStMj7",
        "colab_type": "code",
        "outputId": "b72eb44f-a6a4-4c78-f167-48b52e2e35dc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "# Raw data\n",
        "df = pd.read_csv(args.data_file, header=0)\n",
        "df.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>surname</th>\n",
              "      <th>nationality</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Woodford</td>\n",
              "      <td>English</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Coté</td>\n",
              "      <td>French</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Kore</td>\n",
              "      <td>English</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Koury</td>\n",
              "      <td>Arabic</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Lebzak</td>\n",
              "      <td>Russian</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "    surname nationality\n",
              "0  Woodford     English\n",
              "1      Coté      French\n",
              "2      Kore     English\n",
              "3     Koury      Arabic\n",
              "4    Lebzak     Russian"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "metadata": {
        "id": "pbfVM-YatMnD",
        "colab_type": "code",
        "outputId": "a583b5d1-3125-4b7a-d9d0-dab540c806a8",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 323
        }
      },
      "cell_type": "code",
      "source": [
        "# Split by nationality\n",
        "by_nationality = collections.defaultdict(list)\n",
        "for _, row in df.iterrows():\n",
        "    by_nationality[row.nationality].append(row.to_dict())\n",
        "for nationality in by_nationality:\n",
        "    print (\"{0}: {1}\".format(nationality, len(by_nationality[nationality])))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "English: 2972\n",
            "French: 229\n",
            "Arabic: 1603\n",
            "Russian: 2373\n",
            "Japanese: 775\n",
            "Chinese: 220\n",
            "Italian: 600\n",
            "Czech: 414\n",
            "Irish: 183\n",
            "German: 576\n",
            "Greek: 156\n",
            "Spanish: 258\n",
            "Polish: 120\n",
            "Dutch: 236\n",
            "Vietnamese: 58\n",
            "Korean: 77\n",
            "Portuguese: 55\n",
            "Scottish: 75\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "KdGOoKFjtMpz",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Create split data\n",
        "final_list = []\n",
        "for _, item_list in sorted(by_nationality.items()):\n",
        "    if args.shuffle:\n",
        "        np.random.shuffle(item_list)\n",
        "    n = len(item_list)\n",
        "    n_train = int(args.train_size*n)\n",
        "    n_val = int(args.val_size*n)\n",
        "    n_test = int(args.test_size*n)\n",
        "\n",
        "  # Give data point a split attribute\n",
        "    for item in item_list[:n_train]:\n",
        "        item['split'] = 'train'\n",
        "    for item in item_list[n_train:n_train+n_val]:\n",
        "        item['split'] = 'val'\n",
        "    for item in item_list[n_train+n_val:]:\n",
        "        item['split'] = 'test'  \n",
        "\n",
        "    # Add to final list\n",
        "    final_list.extend(item_list)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "DyDwlzzKtMsz",
        "colab_type": "code",
        "outputId": "2f21d2e1-ffab-4c09-e9e1-bd103cecaca3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "# df with split datasets\n",
        "split_df = pd.DataFrame(final_list)\n",
        "split_df[\"split\"].value_counts()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "train    7680\n",
              "test     1660\n",
              "val      1640\n",
              "Name: split, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "metadata": {
        "id": "17aHMQOwtMvh",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Preprocessing\n",
        "def preprocess_text(text):\n",
        "    text = ' '.join(word.lower() for word in text.split(\" \"))\n",
        "    text = re.sub(r\"([.,!?])\", r\" \\1 \", text)\n",
        "    text = re.sub(r\"[^a-zA-Z.,!?]+\", r\" \", text)\n",
        "    return text\n",
        "    \n",
        "split_df.surname = split_df.surname.apply(preprocess_text)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "wh6D8qfQmS2c",
        "colab_type": "code",
        "outputId": "c28ae3d6-706d-441d-eb92-d447f7295abf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "cell_type": "code",
      "source": [
        "# Save to CSV\n",
        "split_df.to_csv(args.split_data_file, index=False)\n",
        "split_df.head()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>nationality</th>\n",
              "      <th>split</th>\n",
              "      <th>surname</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Arabic</td>\n",
              "      <td>train</td>\n",
              "      <td>bishara</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Arabic</td>\n",
              "      <td>train</td>\n",
              "      <td>nahas</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Arabic</td>\n",
              "      <td>train</td>\n",
              "      <td>ghanem</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Arabic</td>\n",
              "      <td>train</td>\n",
              "      <td>tannous</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Arabic</td>\n",
              "      <td>train</td>\n",
              "      <td>mikhail</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  nationality  split  surname\n",
              "0      Arabic  train  bishara\n",
              "1      Arabic  train    nahas\n",
              "2      Arabic  train   ghanem\n",
              "3      Arabic  train  tannous\n",
              "4      Arabic  train  mikhail"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "metadata": {
        "id": "6nZBgfQTuAA8",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Vocabulary"
      ]
    },
    {
      "metadata": {
        "id": "TeRVQlRZuBgA",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Vocabulary(object):\n",
        "    def __init__(self, token_to_idx=None, add_unk=True, unk_token=\"<UNK>\"):\n",
        "\n",
        "        # Token to index\n",
        "        if token_to_idx is None:\n",
        "            token_to_idx = {}\n",
        "        self.token_to_idx = token_to_idx\n",
        "\n",
        "        # Index to token\n",
        "        self.idx_to_token = {idx: token \\\n",
        "                             for token, idx in self.token_to_idx.items()}\n",
        "        \n",
        "        # Add unknown token\n",
        "        self.add_unk = add_unk\n",
        "        self.unk_token = unk_token\n",
        "        if self.add_unk:\n",
        "            self.unk_index = self.add_token(self.unk_token)\n",
        "\n",
        "    def to_serializable(self):\n",
        "        return {'token_to_idx': self.token_to_idx,\n",
        "                'add_unk': self.add_unk, 'unk_token': self.unk_token}\n",
        "\n",
        "    @classmethod\n",
        "    def from_serializable(cls, contents):\n",
        "        return cls(**contents)\n",
        "\n",
        "    def add_token(self, token):\n",
        "        if token in self.token_to_idx:\n",
        "            index = self.token_to_idx[token]\n",
        "        else:\n",
        "            index = len(self.token_to_idx)\n",
        "            self.token_to_idx[token] = index\n",
        "            self.idx_to_token[index] = token\n",
        "        return index\n",
        "\n",
        "    def add_tokens(self, tokens):\n",
        "        return [self.add_token[token] for token in tokens]\n",
        "\n",
        "    def lookup_token(self, token):\n",
        "        if self.add_unk:\n",
        "            index = self.token_to_idx.get(token, self.unk_index)\n",
        "        else:\n",
        "            index =  self.token_to_idx[token]\n",
        "        return index\n",
        "\n",
        "    def lookup_index(self, index):\n",
        "        if index not in self.idx_to_token:\n",
        "            raise KeyError(\"the index (%d) is not in the Vocabulary\" % index)\n",
        "        return self.idx_to_token[index]\n",
        "\n",
        "    def __str__(self):\n",
        "        return \"<Vocabulary(size=%d)>\" % len(self)\n",
        "\n",
        "    def __len__(self):\n",
        "        return len(self.token_to_idx)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "bH8LMH9wuBi9",
        "colab_type": "code",
        "outputId": "1a9cde73-19a1-4db2-a789-9ecab9e53fa2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "cell_type": "code",
      "source": [
        "# Vocabulary instance\n",
        "nationality_vocab = Vocabulary(add_unk=False)\n",
        "for index, row in df.iterrows():\n",
        "    nationality_vocab.add_token(row.nationality)\n",
        "print (nationality_vocab) # __str__\n",
        "index = nationality_vocab.lookup_token(\"English\")\n",
        "print (index)\n",
        "print (nationality_vocab.lookup_index(index))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<Vocabulary(size=18)>\n",
            "0\n",
            "English\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "57a1lzHPuHHm",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Vectorizer"
      ]
    },
    {
      "metadata": {
        "id": "MwS5BEV-uBlt",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class SurnameVectorizer(object):\n",
        "    def __init__(self, surname_vocab, nationality_vocab):\n",
        "        self.surname_vocab = surname_vocab\n",
        "        self.nationality_vocab = nationality_vocab\n",
        "\n",
        "    def vectorize(self, surname):\n",
        "        one_hot_matrix_size = (len(surname), len(self.surname_vocab))\n",
        "        one_hot_matrix = np.zeros(one_hot_matrix_size, dtype=np.float32)\n",
        "                               \n",
        "        for position_index, character in enumerate(surname):\n",
        "            character_index = self.surname_vocab.lookup_token(character)\n",
        "            one_hot_matrix[position_index][character_index] = 1\n",
        "        \n",
        "        return one_hot_matrix\n",
        "    \n",
        "    def unvectorize(self, one_hot_matrix):\n",
        "        len_name = len(one_hot_matrix)\n",
        "        indices = np.zeros(len_name)\n",
        "        for i in range(len_name):\n",
        "            indices[i] = np.where(one_hot_matrix[i]==1)[0][0]\n",
        "        surname = [self.surname_vocab.lookup_index(index) for index in indices]\n",
        "        return surname\n",
        "\n",
        "    @classmethod\n",
        "    def from_dataframe(cls, df):\n",
        "        surname_vocab = Vocabulary(add_unk=True)\n",
        "        nationality_vocab = Vocabulary(add_unk=False)\n",
        "\n",
        "        # Create vocabularies\n",
        "        for index, row in df.iterrows():\n",
        "            for letter in row.surname: # char-level tokenization\n",
        "                surname_vocab.add_token(letter)\n",
        "            nationality_vocab.add_token(row.nationality)\n",
        "        return cls(surname_vocab, nationality_vocab)\n",
        "\n",
        "    @classmethod\n",
        "    def from_serializable(cls, contents):\n",
        "        surname_vocab = Vocabulary.from_serializable(contents['surname_vocab'])\n",
        "        nationality_vocab =  Vocabulary.from_serializable(contents['nationality_vocab'])\n",
        "        return cls(surname_vocab, nationality_vocab)\n",
        "\n",
        "    def to_serializable(self):\n",
        "        return {'surname_vocab': self.surname_vocab.to_serializable(),\n",
        "                'nationality_vocab': self.nationality_vocab.to_serializable()}"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "zq7RoFAXuBo9",
        "colab_type": "code",
        "outputId": "a8e8f739-5d23-4164-e0e2-fac1b13fe177",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 221
        }
      },
      "cell_type": "code",
      "source": [
        "# Vectorizer instance\n",
        "vectorizer = SurnameVectorizer.from_dataframe(split_df)\n",
        "print (vectorizer.surname_vocab)\n",
        "print (vectorizer.nationality_vocab)\n",
        "vectorized_surname = vectorizer.vectorize(preprocess_text(\"goku\"))\n",
        "print (np.shape(vectorized_surname))\n",
        "print (vectorized_surname)\n",
        "print (vectorizer.unvectorize(vectorized_surname))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<Vocabulary(size=28)>\n",
            "<Vocabulary(size=18)>\n",
            "(4, 28)\n",
            "[[0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
            "  0. 0. 0. 0.]\n",
            " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
            "  0. 0. 0. 0.]\n",
            " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
            "  0. 0. 0. 0.]\n",
            " [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
            "  0. 0. 0. 0.]]\n",
            "['g', 'o', 'k', 'u']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "wwQ8MNp5ZfeG",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Note**: Unlike the bagged ont-hot encoding method in the MLP notebook, we are able to preserve the semantic structure of the surnames. We are able to use one-hot encoding here because we are using characters but when we process text with large vocabularies, this method simply can't scale. We'll explore embedding based methods in subsequent notebooks. "
      ]
    },
    {
      "metadata": {
        "id": "Mnf7gXgKuOgp",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Dataset"
      ]
    },
    {
      "metadata": {
        "id": "YYqzM53fuBrf",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "from torch.utils.data import Dataset, DataLoader"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "gjolk855uPrA",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class SurnameDataset(Dataset):\n",
        "    def __init__(self, df, vectorizer):\n",
        "        self.df = df\n",
        "        self.vectorizer = vectorizer\n",
        "\n",
        "        # Data splits\n",
        "        self.train_df = self.df[self.df.split=='train']\n",
        "        self.train_size = len(self.train_df)\n",
        "        self.val_df = self.df[self.df.split=='val']\n",
        "        self.val_size = len(self.val_df)\n",
        "        self.test_df = self.df[self.df.split=='test']\n",
        "        self.test_size = len(self.test_df)\n",
        "        self.lookup_dict = {'train': (self.train_df, self.train_size), \n",
        "                            'val': (self.val_df, self.val_size),\n",
        "                            'test': (self.test_df, self.test_size)}\n",
        "        self.set_split('train')\n",
        "\n",
        "        # Class weights (for imbalances)\n",
        "        class_counts = df.nationality.value_counts().to_dict()\n",
        "        def sort_key(item):\n",
        "            return self.vectorizer.nationality_vocab.lookup_token(item[0])\n",
        "        sorted_counts = sorted(class_counts.items(), key=sort_key)\n",
        "        frequencies = [count for _, count in sorted_counts]\n",
        "        self.class_weights = 1.0 / torch.tensor(frequencies, dtype=torch.float32)\n",
        "\n",
        "    @classmethod\n",
        "    def load_dataset_and_make_vectorizer(cls, split_data_file):\n",
        "        df = pd.read_csv(split_data_file, header=0)\n",
        "        train_df = df[df.split=='train']\n",
        "        return cls(df, SurnameVectorizer.from_dataframe(train_df))\n",
        "\n",
        "    @classmethod\n",
        "    def load_dataset_and_load_vectorizer(cls, split_data_file, vectorizer_filepath):\n",
        "        df = pd.read_csv(split_data_file, header=0)\n",
        "        vectorizer = cls.load_vectorizer_only(vectorizer_filepath)\n",
        "        return cls(df, vectorizer)\n",
        "\n",
        "    def load_vectorizer_only(vectorizer_filepath):\n",
        "        with open(vectorizer_filepath) as fp:\n",
        "            return SurnameVectorizer.from_serializable(json.load(fp))\n",
        "\n",
        "    def save_vectorizer(self, vectorizer_filepath):\n",
        "        with open(vectorizer_filepath, \"w\") as fp:\n",
        "            json.dump(self.vectorizer.to_serializable(), fp)\n",
        "\n",
        "    def set_split(self, split=\"train\"):\n",
        "        self.target_split = split\n",
        "        self.target_df, self.target_size = self.lookup_dict[split]\n",
        "\n",
        "    def __str__(self):\n",
        "        return \"<Dataset(split={0}, size={1})\".format(\n",
        "            self.target_split, self.target_size)\n",
        "\n",
        "    def __len__(self):\n",
        "        return self.target_size\n",
        "\n",
        "    def __getitem__(self, index):\n",
        "        row = self.target_df.iloc[index]\n",
        "        surname_vector = self.vectorizer.vectorize(row.surname)\n",
        "        nationality_index = self.vectorizer.nationality_vocab.lookup_token(row.nationality)\n",
        "        return {'surname': surname_vector, 'nationality': nationality_index}\n",
        "\n",
        "    def get_num_batches(self, batch_size):\n",
        "        return len(self) // batch_size\n",
        "\n",
        "    def generate_batches(self, batch_size, collate_fn, shuffle=True, \n",
        "                         drop_last=True, device=\"cpu\"):\n",
        "        dataloader = DataLoader(dataset=self, batch_size=batch_size,\n",
        "                                collate_fn=collate_fn, shuffle=shuffle, \n",
        "                                drop_last=drop_last)\n",
        "        for data_dict in dataloader:\n",
        "            out_data_dict = {}\n",
        "            for name, tensor in data_dict.items():\n",
        "                out_data_dict[name] = data_dict[name].to(device)\n",
        "            yield out_data_dict"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "hvy-CJVSuPuS",
        "colab_type": "code",
        "outputId": "962f8b0d-116b-4827-b9be-6ca25104858d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "cell_type": "code",
      "source": [
        "# Dataset instance\n",
        "dataset = SurnameDataset.load_dataset_and_make_vectorizer(args.split_data_file)\n",
        "print (dataset) # __str__\n",
        "print (np.shape(dataset[5]['surname'])) # __getitem__\n",
        "print (dataset.class_weights)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<Dataset(split=train, size=7680)\n",
            "(6, 28)\n",
            "tensor([0.0006, 0.0045, 0.0024, 0.0042, 0.0003, 0.0044, 0.0017, 0.0064, 0.0055,\n",
            "        0.0017, 0.0013, 0.0130, 0.0083, 0.0182, 0.0004, 0.0133, 0.0039, 0.0172])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "XY0CqM2Rd3Im",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Model"
      ]
    },
    {
      "metadata": {
        "id": "pWGpAzKPd32f",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch.nn as nn\n",
        "import torch.nn.functional as F"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "d7Q0_nkjd30L",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class SurnameModel(nn.Module):\n",
        "    def __init__(self, num_input_channels, num_output_channels, num_classes, dropout_p):\n",
        "        super(SurnameModel, self).__init__()\n",
        "        \n",
        "        # Conv weights\n",
        "        self.conv = nn.ModuleList([nn.Conv1d(num_input_channels, num_output_channels, \n",
        "                                             kernel_size=f) for f in [2,3,4]])\n",
        "        self.dropout = nn.Dropout(dropout_p)\n",
        "       \n",
        "        # FC weights\n",
        "        self.fc1 = nn.Linear(num_output_channels*3, num_classes)\n",
        "\n",
        "    def forward(self, x, channel_first=False, apply_softmax=False):\n",
        "        \n",
        "        # Rearrange input so num_input_channels is in dim 1 (N, C, L)\n",
        "        if not channel_first:\n",
        "            x = x.transpose(1, 2)\n",
        "            \n",
        "        # Conv outputs\n",
        "        z = [conv(x) for conv in self.conv]\n",
        "        z = [F.max_pool1d(zz, zz.size(2)).squeeze(2) for zz in z]\n",
        "        z = [F.relu(zz) for zz in z]\n",
        "        \n",
        "        # Concat conv outputs\n",
        "        z = torch.cat(z, 1)\n",
        "        z = self.dropout(z)\n",
        "\n",
        "        # FC layer\n",
        "        y_pred = self.fc1(z)\n",
        "        \n",
        "        if apply_softmax:\n",
        "            y_pred = F.softmax(y_pred, dim=1)\n",
        "        return y_pred"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "7XlJwSKQkL_C",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Training"
      ]
    },
    {
      "metadata": {
        "id": "rh_1heUNSUYN",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "**Padding:** the inputs in a particular batch must all have the same shape. Our vectorizer converts the tokens into a vectorizer form but in a particular batch, we can have inputs of various sizes. The solution is to determine the longest input in a particular batch and pad all the other inputs to match that length. Usually, the smaller inputs in the batch are padded with zero vectors. \n",
        "\n",
        "We do this using the pad_seq function in the Trainer class which is invoked by the collate_fn which is passed to generate_batches function in the Dataset class. Essentially, the batch generater generates samples into a batch and we use the collate_fn to determine the largest input and pad all the other inputs in the batch to get a uniform input shape."
      ]
    },
    {
      "metadata": {
        "id": "wLLmIuKRkNYW",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "import torch.optim as optim"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "sV-Dc_5ykNgS",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Trainer(object):\n",
        "    def __init__(self, dataset, model, model_state_file, save_dir, device, shuffle, \n",
        "               num_epochs, batch_size, learning_rate, early_stopping_criteria):\n",
        "        self.dataset = dataset\n",
        "        self.class_weights = dataset.class_weights.to(device)\n",
        "        self.model = model.to(device)\n",
        "        self.save_dir = save_dir\n",
        "        self.device = device\n",
        "        self.shuffle = shuffle\n",
        "        self.num_epochs = num_epochs\n",
        "        self.batch_size = batch_size\n",
        "        self.loss_func = nn.CrossEntropyLoss(self.class_weights)\n",
        "        self.optimizer = optim.Adam(self.model.parameters(), lr=learning_rate)\n",
        "        self.scheduler = optim.lr_scheduler.ReduceLROnPlateau(\n",
        "            optimizer=self.optimizer, mode='min', factor=0.5, patience=1)\n",
        "        self.train_state = {\n",
        "            'stop_early': False, \n",
        "            'early_stopping_step': 0,\n",
        "            'early_stopping_best_val': 1e8,\n",
        "            'early_stopping_criteria': early_stopping_criteria,\n",
        "            'learning_rate': learning_rate,\n",
        "            'epoch_index': 0,\n",
        "            'train_loss': [],\n",
        "            'train_acc': [],\n",
        "            'val_loss': [],\n",
        "            'val_acc': [],\n",
        "            'test_loss': -1,\n",
        "            'test_acc': -1,\n",
        "            'model_filename': model_state_file}\n",
        "    \n",
        "    def update_train_state(self):\n",
        "\n",
        "        # Verbose\n",
        "        print (\"[EPOCH]: {0:02d} | [LR]: {1} | [TRAIN LOSS]: {2:.2f} | [TRAIN ACC]: {3:.1f}% | [VAL LOSS]: {4:.2f} | [VAL ACC]: {5:.1f}%\".format(\n",
        "          self.train_state['epoch_index'], self.train_state['learning_rate'], \n",
        "            self.train_state['train_loss'][-1], self.train_state['train_acc'][-1], \n",
        "            self.train_state['val_loss'][-1], self.train_state['val_acc'][-1]))\n",
        "\n",
        "        # Save one model at least\n",
        "        if self.train_state['epoch_index'] == 0:\n",
        "            torch.save(self.model.state_dict(), self.train_state['model_filename'])\n",
        "            self.train_state['stop_early'] = False\n",
        "\n",
        "        # Save model if performance improved\n",
        "        elif self.train_state['epoch_index'] >= 1:\n",
        "            loss_tm1, loss_t = self.train_state['val_loss'][-2:]\n",
        "\n",
        "            # If loss worsened\n",
        "            if loss_t >= self.train_state['early_stopping_best_val']:\n",
        "                # Update step\n",
        "                self.train_state['early_stopping_step'] += 1\n",
        "\n",
        "            # Loss decreased\n",
        "            else:\n",
        "                # Save the best model\n",
        "                if loss_t < self.train_state['early_stopping_best_val']:\n",
        "                    torch.save(self.model.state_dict(), self.train_state['model_filename'])\n",
        "\n",
        "                # Reset early stopping step\n",
        "                self.train_state['early_stopping_step'] = 0\n",
        "\n",
        "            # Stop early ?\n",
        "            self.train_state['stop_early'] = self.train_state['early_stopping_step'] \\\n",
        "              >= self.train_state['early_stopping_criteria']\n",
        "        return self.train_state\n",
        "  \n",
        "    def compute_accuracy(self, y_pred, y_target):\n",
        "        _, y_pred_indices = y_pred.max(dim=1)\n",
        "        n_correct = torch.eq(y_pred_indices, y_target).sum().item()\n",
        "        return n_correct / len(y_pred_indices) * 100\n",
        "    \n",
        "    def pad_seq(self, seq, length):\n",
        "        vector = np.zeros((length, len(self.dataset.vectorizer.surname_vocab)),\n",
        "                          dtype=np.int64)\n",
        "        for i in range(len(seq)):\n",
        "            vector[i] = seq[i]\n",
        "        return vector\n",
        "    \n",
        "    def collate_fn(self, batch):\n",
        "        \n",
        "        # Make a deep copy\n",
        "        batch_copy = copy.deepcopy(batch)\n",
        "        processed_batch = {\"surname\": [], \"nationality\": []}\n",
        "        \n",
        "        # Get max sequence length\n",
        "        max_seq_len = max([len(sample[\"surname\"]) for sample in batch_copy])\n",
        "        \n",
        "        # Pad\n",
        "        for i, sample in enumerate(batch_copy):\n",
        "            seq = sample[\"surname\"]\n",
        "            nationality = sample[\"nationality\"]\n",
        "            padded_seq = self.pad_seq(seq, max_seq_len)\n",
        "            processed_batch[\"surname\"].append(padded_seq)\n",
        "            processed_batch[\"nationality\"].append(nationality)\n",
        "            \n",
        "        # Convert to appropriate tensor types\n",
        "        processed_batch[\"surname\"] = torch.FloatTensor(\n",
        "            processed_batch[\"surname\"]) # need float for conv operations\n",
        "        processed_batch[\"nationality\"] = torch.LongTensor(\n",
        "            processed_batch[\"nationality\"])\n",
        "        \n",
        "        return processed_batch    \n",
        "  \n",
        "    def run_train_loop(self):\n",
        "        for epoch_index in range(self.num_epochs):\n",
        "            self.train_state['epoch_index'] = epoch_index\n",
        "      \n",
        "            # Iterate over train dataset\n",
        "\n",
        "            # initialize batch generator, set loss and acc to 0, set train mode on\n",
        "            self.dataset.set_split('train')\n",
        "            batch_generator = self.dataset.generate_batches(\n",
        "                batch_size=self.batch_size, collate_fn=self.collate_fn,\n",
        "                shuffle=self.shuffle, device=self.device)\n",
        "            running_loss = 0.0\n",
        "            running_acc = 0.0\n",
        "            self.model.train()\n",
        "\n",
        "            for batch_index, batch_dict in enumerate(batch_generator):\n",
        "                # zero the gradients\n",
        "                self.optimizer.zero_grad()\n",
        "\n",
        "                # compute the output\n",
        "                y_pred = self.model(batch_dict['surname'])\n",
        "\n",
        "                # compute the loss\n",
        "                loss = self.loss_func(y_pred, batch_dict['nationality'])\n",
        "                loss_t = loss.item()\n",
        "                running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
        "\n",
        "                # compute gradients using loss\n",
        "                loss.backward()\n",
        "\n",
        "                # use optimizer to take a gradient step\n",
        "                self.optimizer.step()\n",
        "                \n",
        "                # compute the accuracy\n",
        "                acc_t = self.compute_accuracy(y_pred, batch_dict['nationality'])\n",
        "                running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
        "\n",
        "            self.train_state['train_loss'].append(running_loss)\n",
        "            self.train_state['train_acc'].append(running_acc)\n",
        "\n",
        "            # Iterate over val dataset\n",
        "\n",
        "            # initialize batch generator, set loss and acc to 0; set eval mode on\n",
        "            self.dataset.set_split('val')\n",
        "            batch_generator = self.dataset.generate_batches(\n",
        "                batch_size=self.batch_size, collate_fn=self.collate_fn, \n",
        "                shuffle=self.shuffle, device=self.device)\n",
        "            running_loss = 0.\n",
        "            running_acc = 0.\n",
        "            self.model.eval()\n",
        "\n",
        "            for batch_index, batch_dict in enumerate(batch_generator):\n",
        "\n",
        "                # compute the output\n",
        "                y_pred =  self.model(batch_dict['surname'])\n",
        "\n",
        "                # compute the loss\n",
        "                loss = self.loss_func(y_pred, batch_dict['nationality'])\n",
        "                loss_t = loss.to(\"cpu\").item()\n",
        "                running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
        "\n",
        "                # compute the accuracy\n",
        "                acc_t = self.compute_accuracy(y_pred, batch_dict['nationality'])\n",
        "                running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
        "\n",
        "            self.train_state['val_loss'].append(running_loss)\n",
        "            self.train_state['val_acc'].append(running_acc)\n",
        "\n",
        "            self.train_state = self.update_train_state()\n",
        "            self.scheduler.step(self.train_state['val_loss'][-1])\n",
        "            if self.train_state['stop_early']:\n",
        "                break\n",
        "          \n",
        "    def run_test_loop(self):\n",
        "        # initialize batch generator, set loss and acc to 0; set eval mode on\n",
        "        self.dataset.set_split('test')\n",
        "        batch_generator = self.dataset.generate_batches(\n",
        "            batch_size=self.batch_size, collate_fn=self.collate_fn,\n",
        "            shuffle=self.shuffle, device=self.device)\n",
        "        running_loss = 0.0\n",
        "        running_acc = 0.0\n",
        "        self.model.eval()\n",
        "\n",
        "        for batch_index, batch_dict in enumerate(batch_generator):\n",
        "            # compute the output\n",
        "            y_pred =  self.model(batch_dict['surname'])\n",
        "\n",
        "            # compute the loss\n",
        "            loss = self.loss_func(y_pred, batch_dict['nationality'])\n",
        "            loss_t = loss.item()\n",
        "            running_loss += (loss_t - running_loss) / (batch_index + 1)\n",
        "\n",
        "            # compute the accuracy\n",
        "            acc_t = self.compute_accuracy(y_pred, batch_dict['nationality'])\n",
        "            running_acc += (acc_t - running_acc) / (batch_index + 1)\n",
        "\n",
        "        self.train_state['test_loss'] = running_loss\n",
        "        self.train_state['test_acc'] = running_acc\n",
        "    \n",
        "    def plot_performance(self):\n",
        "        # Figure size\n",
        "        plt.figure(figsize=(15,5))\n",
        "\n",
        "        # Plot Loss\n",
        "        plt.subplot(1, 2, 1)\n",
        "        plt.title(\"Loss\")\n",
        "        plt.plot(trainer.train_state[\"train_loss\"], label=\"train\")\n",
        "        plt.plot(trainer.train_state[\"val_loss\"], label=\"val\")\n",
        "        plt.legend(loc='upper right')\n",
        "\n",
        "        # Plot Accuracy\n",
        "        plt.subplot(1, 2, 2)\n",
        "        plt.title(\"Accuracy\")\n",
        "        plt.plot(trainer.train_state[\"train_acc\"], label=\"train\")\n",
        "        plt.plot(trainer.train_state[\"val_acc\"], label=\"val\")\n",
        "        plt.legend(loc='lower right')\n",
        "\n",
        "        # Save figure\n",
        "        plt.savefig(os.path.join(self.save_dir, \"performance.png\"))\n",
        "\n",
        "        # Show plots\n",
        "        plt.show()\n",
        "    \n",
        "    def save_train_state(self):\n",
        "        with open(os.path.join(self.save_dir, \"train_state.json\"), \"w\") as fp:\n",
        "            json.dump(self.train_state, fp)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "OkeOQRwckNd1",
        "colab_type": "code",
        "outputId": "92bf0944-5cae-4e91-8d9d-ae0ebccec738",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        }
      },
      "cell_type": "code",
      "source": [
        "# Initialization\n",
        "dataset = SurnameDataset.load_dataset_and_make_vectorizer(args.split_data_file)\n",
        "dataset.save_vectorizer(args.vectorizer_file)\n",
        "vectorizer = dataset.vectorizer\n",
        "model = SurnameModel(num_input_channels=len(vectorizer.surname_vocab),\n",
        "                     num_output_channels=args.num_filters,\n",
        "                     num_classes=len(vectorizer.nationality_vocab),\n",
        "                     dropout_p=args.dropout_p)\n",
        "print (model.named_modules)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Creating from scratch!\n",
            "<bound method Module.named_modules of SurnameModel(\n",
            "  (conv): ModuleList(\n",
            "    (0): Conv1d(28, 100, kernel_size=(2,), stride=(1,))\n",
            "    (1): Conv1d(28, 100, kernel_size=(3,), stride=(1,))\n",
            "    (2): Conv1d(28, 100, kernel_size=(4,), stride=(1,))\n",
            "  )\n",
            "  (dropout): Dropout(p=0.1)\n",
            "  (fc1): Linear(in_features=300, out_features=18, bias=True)\n",
            ")>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "3JJdOO4ZkNb3",
        "colab_type": "code",
        "outputId": "1bc45f26-92d9-421b-bf1d-53971fa177de",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        }
      },
      "cell_type": "code",
      "source": [
        "# Train\n",
        "trainer = Trainer(dataset=dataset, model=model, \n",
        "                  model_state_file=args.model_state_file, \n",
        "                  save_dir=args.save_dir, device=args.device,\n",
        "                  shuffle=args.shuffle, num_epochs=args.num_epochs, \n",
        "                  batch_size=args.batch_size, learning_rate=args.learning_rate, \n",
        "                  early_stopping_criteria=args.early_stopping_criteria)\n",
        "trainer.run_train_loop()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[EPOCH]: 00 | [LR]: 0.001 | [TRAIN LOSS]: 2.82 | [TRAIN ACC]: 21.2% | [VAL LOSS]: 2.73 | [VAL ACC]: 41.0%\n",
            "[EPOCH]: 01 | [LR]: 0.001 | [TRAIN LOSS]: 2.55 | [TRAIN ACC]: 43.6% | [VAL LOSS]: 2.41 | [VAL ACC]: 50.2%\n",
            "[EPOCH]: 02 | [LR]: 0.001 | [TRAIN LOSS]: 2.19 | [TRAIN ACC]: 49.2% | [VAL LOSS]: 2.13 | [VAL ACC]: 49.6%\n",
            "[EPOCH]: 03 | [LR]: 0.001 | [TRAIN LOSS]: 1.90 | [TRAIN ACC]: 51.8% | [VAL LOSS]: 1.93 | [VAL ACC]: 57.1%\n",
            "[EPOCH]: 04 | [LR]: 0.001 | [TRAIN LOSS]: 1.69 | [TRAIN ACC]: 55.2% | [VAL LOSS]: 1.78 | [VAL ACC]: 52.0%\n",
            "[EPOCH]: 05 | [LR]: 0.001 | [TRAIN LOSS]: 1.52 | [TRAIN ACC]: 57.4% | [VAL LOSS]: 1.66 | [VAL ACC]: 58.5%\n",
            "[EPOCH]: 06 | [LR]: 0.001 | [TRAIN LOSS]: 1.38 | [TRAIN ACC]: 60.6% | [VAL LOSS]: 1.59 | [VAL ACC]: 56.7%\n",
            "[EPOCH]: 07 | [LR]: 0.001 | [TRAIN LOSS]: 1.29 | [TRAIN ACC]: 62.1% | [VAL LOSS]: 1.51 | [VAL ACC]: 55.2%\n",
            "[EPOCH]: 08 | [LR]: 0.001 | [TRAIN LOSS]: 1.20 | [TRAIN ACC]: 64.0% | [VAL LOSS]: 1.48 | [VAL ACC]: 59.9%\n",
            "[EPOCH]: 09 | [LR]: 0.001 | [TRAIN LOSS]: 1.12 | [TRAIN ACC]: 65.3% | [VAL LOSS]: 1.43 | [VAL ACC]: 59.9%\n",
            "[EPOCH]: 10 | [LR]: 0.001 | [TRAIN LOSS]: 1.06 | [TRAIN ACC]: 65.5% | [VAL LOSS]: 1.38 | [VAL ACC]: 61.2%\n",
            "[EPOCH]: 11 | [LR]: 0.001 | [TRAIN LOSS]: 0.99 | [TRAIN ACC]: 67.0% | [VAL LOSS]: 1.38 | [VAL ACC]: 62.6%\n",
            "[EPOCH]: 12 | [LR]: 0.001 | [TRAIN LOSS]: 0.94 | [TRAIN ACC]: 68.5% | [VAL LOSS]: 1.35 | [VAL ACC]: 66.1%\n",
            "[EPOCH]: 13 | [LR]: 0.001 | [TRAIN LOSS]: 0.90 | [TRAIN ACC]: 69.1% | [VAL LOSS]: 1.37 | [VAL ACC]: 63.7%\n",
            "[EPOCH]: 14 | [LR]: 0.001 | [TRAIN LOSS]: 0.85 | [TRAIN ACC]: 69.7% | [VAL LOSS]: 1.30 | [VAL ACC]: 64.9%\n",
            "[EPOCH]: 15 | [LR]: 0.001 | [TRAIN LOSS]: 0.81 | [TRAIN ACC]: 70.4% | [VAL LOSS]: 1.28 | [VAL ACC]: 66.6%\n",
            "[EPOCH]: 16 | [LR]: 0.001 | [TRAIN LOSS]: 0.79 | [TRAIN ACC]: 71.4% | [VAL LOSS]: 1.29 | [VAL ACC]: 66.4%\n",
            "[EPOCH]: 17 | [LR]: 0.001 | [TRAIN LOSS]: 0.74 | [TRAIN ACC]: 71.7% | [VAL LOSS]: 1.30 | [VAL ACC]: 67.5%\n",
            "[EPOCH]: 18 | [LR]: 0.001 | [TRAIN LOSS]: 0.70 | [TRAIN ACC]: 73.2% | [VAL LOSS]: 1.27 | [VAL ACC]: 68.4%\n",
            "[EPOCH]: 19 | [LR]: 0.001 | [TRAIN LOSS]: 0.69 | [TRAIN ACC]: 73.3% | [VAL LOSS]: 1.31 | [VAL ACC]: 67.6%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "0QLZfEyznVpT",
        "colab_type": "code",
        "outputId": "0f436371-3b96-4e59-a0fd-6868688c59e9",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 335
        }
      },
      "cell_type": "code",
      "source": [
        "# Plot performance\n",
        "trainer.plot_performance()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2gAAAE+CAYAAAD4XjP+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XdYHOe5///3FpbeYekdREcN1JBA\nQr1Z1Zbt2HHsNJ9fEifn+OvjJMcnTuJUJ05ziU8s19hxZMu2LEuyVVFDDUlIAoFE723pve3u7w9k\nJKwuQMvC/bouXezOzM58Fha0994zz6MwGo1GhBBCCCGEEEKYnNLUAYQQQgghhBBC9JMCTQghhBBC\nCCFGCSnQhBBCCCGEEGKUkAJNCCGEEEIIIUYJKdCEEEIIIYQQYpSQAk0IIYQQQgghRgkp0IS4Q+Hh\n4VRXV5s6hhBCCHFX3H///dxzzz2mjiHEmCcFmhBCCCGEuKHc3Fzs7e3x9vYmIyPD1HGEGNOkQBNi\nmHV3d/Ozn/2MxYsXs3TpUn73u9+h1+sBePfdd1m6dClLlixh/fr15OXl3XC5EEIIMRp88sknLFmy\nhBUrVrBly5aB5Vu2bGHx4sUsXryYp556ip6enusuP378OAsXLhx47JX3X3zxRZ555hnWr1/PW2+9\nhcFg4Be/+AWLFy8mJSWFp556it7eXgAaGhp4/PHHmT9/PitXruTw4cPs37+fFStWDMq8du1a9uzZ\nM9LfGiGGndrUAYQYa95++22qq6vZvn07fX19PPTQQ2zbto358+fz17/+ldTUVOzs7Pj888/Zv38/\nXl5e11weFhZm6qcihBBCoNfr2b17N9/73vdQqVS88MIL9PT0UFtby+9//3u2bNmCVqvlBz/4Ae+8\n8w5Lliy55vLY2NgbHufAgQN8+umnuLi4sHPnTk6ePMm2bdswGAysWbOGHTt2sGrVKl544QVCQkJ4\n9dVXyc7O5tFHH+XQoUPodDouXLhAREQElZWVlJaWkpSUdJe+S0IMHynQhBhm+/fv57HHHkOtVqNW\nq1m5ciVpaWksW7YMhULB5s2bWbFiBUuXLgWgt7f3msuFEEKI0eDw4cPExsZiZ2cHwLRp00hNTaWp\nqYnJkyfj4eEBwAsvvIBKpeKjjz665vJTp07d8DgTJ07ExcUFgMWLFzNv3jwsLCwAiI2NpaysDOgv\n5F577TUAoqKi2Lt3LxqNhsWLF7N9+3YiIiLYs2cP8+fPR6PRDP83RIgRJqc4CjHMGhoacHR0HLjv\n6OhIfX09FhYWvPXWW5w+fZrFixfz4IMPcvHixesuF0IIIUaDjz/+mP379xMfH098fDy7du3ik08+\nobGxEQcHh4HtLC0tUavV111+M1f+39nQ0MDTTz/N4sWLWbJkCXv37sVoNALQ1NSEvb39wLZfFo7L\nly9n+/btAOzZs4dly5YN7YkLYSJSoAkxzNzc3Ghqahq439TUhJubG9D/Sd/f/vY3jh49yuzZs3n2\n2WdvuFwIIYQwpebmZk6cOMHx48c5efIkJ0+eJD09nczMTJRKJY2NjQPbtrW1UVdXh7Oz8zWXq1Sq\ngWuyAVpaWq573D//+c+o1Wo+++wzvvjiC5KTkwfWOTk5Ddp/eXk5vb29JCQk0NfXR2pqKnl5ecya\nNWu4vg1C3FVSoAkxzObOncvmzZvR6/V0dHTw6aefkpyczMWLF3niiSfo6elBo9EQExODQqG47nIh\nhBDC1LZv386MGTMGnSqoVquZPXs2PT09nD59mvLycoxGI88++yybN28mOTn5msvd3d3R6XTU19ej\n1+v57LPPrnvc+vp6JkyYgEaj4cKFC2RkZNDR0QFASkoKn3zyCQD5+fmsXbsWvV6PUqlk2bJlPPfc\nc6SkpAycHimEuZFr0IQYgocffhiVSjVw/1e/+hUPP/wwZWVlLF++HIVCwZIlSwauK/P19WXFihVY\nWFhga2vLz372MyZMmHDN5UIIIYSpbdmyhUceeeSq5QsXLuSVV17hl7/8JY888ggqlYrY2FgeffRR\nLC0tr7t83bp1rF69Gm9vb1atWkVOTs41j/vYY4/x9NNP8/HHHxMfH8/TTz/N//zP/xAXF8dTTz3F\n008/TUpKCra2tvzxj3/EysoK6D/N8c0335TTG4VZUxi/PKFXCCGEEEIIM1ZXV8eaNWvYv3//oA9Q\nhTAncoqjEEIIIYQYE/72t7/xwAMPSHEmzJoUaEIIIYQQwqzV1dUxf/586urqeOyxx0wdR4ghkVMc\nhRBCCCGEEGKUkA6aEEIIIYQQQowSUqAJIYQQQgghxChx14fZ1+lah7wPZ2cbGhs7hiHN3SW57x5z\nzAzmmdscM4N55jbHzO7u9qaOYFbk/0jzym2OmcE8c5tjZjDP3OaYGcwv943+fzTLDppabZ4j80ju\nu8ccM4N55jbHzGCeuc0xs7j7zPV1Yo65zTEzmGduc8wM5pnbHDOD+ea+FrMs0IQQQgghhBBiLJIC\nTQghhBBCCCFGCSnQhBBCCCGEEGKUkAJNCCGEEEIIIUYJKdCEEEIIIYQQYpSQAk0IIYQQQgghRgkp\n0IQQQgghhBBilJACTQghxpH9+/fe0nZ//esLVFZWjHAaIYQQQnyVFGhCCDFOVFVVsmfPzlva9oc/\nfBJvb58RTiSEEEKIr1KbOsDt6u7R89mhQiYFO2OlMbv4QghhMn/60+/JyTnPnDkJLFq0lKqqSv7y\nl1f47W9/iU5XS2dnJ4899h0SE+fw/e9/h//6r/8mNXUv7e1tlJaWUFFRzhNPPMnMmYmmfipCCCHE\nLTMYjRRVtdDe2YvB0H/fYDBiMBrRGy7dvnS//yuXlxuv/jop1J1gb4cRy2t2Fc7Fskb+sSWTeVN8\neHhRuKnjCCGE2XjggYf5+OMPCAoKobS0mFde2UhjYwPTps1g6dIVVFSU87//+2MSE+cMelxtbQ1/\n/OPfOHbsCJ9++pEUaEIIIcxCc3sPaZlVHDxTSW1T57Dtt6q+g++tiR22/X2V2RVoUYEu+LjbciCj\nkpQpvvi42Zo6khBC3LYP9uWTfqH2th6jUinQ643XXZ8QoeW+lNBb2ldkZDQA9vYO5OScZ+vWj1Eo\nlLS0NF+1bVzcJAC0Wi1tbW23lVkIIYS4mwxGI9lFDRw4W8mZvDr0BiMatZJZMZ54udqgUipRKkCp\nVFz+p1CguvT1ymVKJYOXX/rq72E3os/B7Ao0tUrJYytjeO6N42zal8d/3TfJ1JGEEMLsWFhYALB7\n9xe0tLTw8ssbaWlp4VvfeviqbVUq1cBto/H6BaIQQghhKvXNnXx2pJhDZyupa+4CwNfdjuRJ3syM\n9sDGysLECW+d2RVoAAlRHkQGOJNV2EBmYT2xwa6mjiSEELflvpTQW+52fcnd3R6drvWOj6lUKtHr\n9YOWNTU14eXljVKp5MCBffT29t7x/oUQQoi7yWAwklVUz4EzlZwtqMdgMKKxUDInzovkST4Eedmj\nUChMHfO2mWWBplAouH9+GD9/8wSb9uUTFeiMSikDUgohxI0EBARx8eIFvLy8cXJyAmDu3BR+/OP/\nIjs7i+XL70Gr1fLmm6+ZOKkQQghxfQ0tXRw+V8XBc5U0tHQDEOzjSGKMJzOiPLC2NMsSZ4DZpvfT\n2jEnzpuDZys5cKb/ejQhhBDX5+zszMcfbx+0zMvLm7ff/vfA/UWLlgLw6KPfBiA4+HKXLzg4lJde\n+sddSCqEEEIMpjcYyCxo4MCZCs4V1mM0gqVGRfIkb5IneZMQ6zOks0xGE7Mt0ADWJAVzIqeGLYeK\nmBFlXueWCiGEEEIIIa7PaDRSf6lbduhcFY2t/d2yIC97kif5MC1SOyan3TLrZ+Roq2H5zAA+OlDI\nZ0eK2ZASZupIQgghhBBCiCv09hno6OqlvauP9ktfO7p6ae/8yv1L6zu6+vpvd/aiN/QPTmVtqWLe\nFB+SJ3rj72Fv4mc0ssy6QANYlODHgTOV7DlZztzJPng425g6khBCCCGEEONOY2s32cUN5JQ0UlrT\nOlBw9fQabnkfSoUCGys1tlZq3BytcLDRMHmCG9MiPLDUqG6+gzHA7As0C7WK9XNDePXT83yYWsD3\n147cpHFCCCGEEEKIfu1dvVwoaSS7pJGc4kaqGzoG1llaqLC3scDLxba/4LK2wNZKja1V/1ebK27b\nWlsM3LfSqMxy5MXhZHYFmtFopLK1Bgsud8oSIrTsOVXO6VwdF0sbCfd3NmFCIYQQQgghxp7uXj15\n5U3kFPcXZaXVrXw5O6alhYq4EFci/J2JCnTGV2uHcpwXWnfK7Aq0C415vJS6kQcj1pHoPR3oH3b/\ngflhPPf2Sf69N5///Ua8vCCEEEIIIYQYgj69geKqVrJLGsgpbqSgspk+fX9JplIqCPNzIirAmchA\nZ4K8HFCrZNqr4WB2BZq3rRcWKgs+L9rLNM+pWCj7n0KQlwMzoz04er6GI5nVzI7zMnFSIYQwT+vX\nr+SddzZhYyPX9N6qDz/8kK1btw7cz8rK4v333+fnP/85AOHh4fziF78wUTohhLg1BqORospmjmSU\nk13SyMWyJrp79AAoAH8PeyIDnYkKcCbM12ncXBN2t5ldgeZoac+ikCS25+7lWFU6c3xmDqxblxzC\nqYs6PjpYQHyE+5gcdlMIIcToc++993LvvfcCcOLECT7//HN+/etf89Of/pS4uDiefPJJDhw4QHJy\nsomTCiFEv94+PRV17ZTWtFFW00ZpbSvlujY6u/UD23i42PR3yAKciQhwxs5aprS6G8yyglkVsZBd\n+QfZWZzKDK+EgS6ai4MVS6b7szWtmM+PlbImKdjESYUQYvR47LGv8ZvfvICnpyfV1VX85CdP4u6u\npbOzk66uLv7zP58iKirG1DHN3ssvv8xvf/tbHnroIeLi4gCYN28eR48elQJNCGESLR09lNVeLsTK\natqoqu/AYDQObKNQgJerLTNjXQjysCMywBkXBysTph6/zLJAc7J2ZI7PDPaVHbqqi7Z0egAHz1ay\n80QpyZO85YUlhBCXJCXNIy3tIOvW3cehQwdISppHSEgYSUlzOXUqnffee5tf//oPpo5p1s6dO4eX\nlxcqlQoHB4eB5a6uruh0OhMmE0KMBwajkdrGTspq2yitaR342tTWM2g7S42KYB8H/LV2+HvY46e1\nw8fNFo2FCnd3e3S6VhM9AwFmWqABLPCfy6GKY3xRvG9QF81So2Jdcgivb89h84ECvrMy2sRJhRDi\nah/nbyOjNvO2HqNSKgYm7LyWydpY1oauuO76pKR5vPTSX1i37j4OHz7A97//n/z73//k/ff/SW9v\nL1ZW8oHWUG3evJk1a9ZctdxovP7P7UrOzjao1UO/psPd3TwncTXH3OaYGcwztzlmhpHJbTAY0TV1\nUl7bSnltG+W1bRRXNlNc1UJXj37Qtm6OViREeRDk7UiwtyNBPg54utiiVF5/QD35XpuW2RZojpb2\nA120o5XpJPle7qLNjPFkz8lyjp2vYcFUP4K9HW6wJyGEGB+Cg0Oor9dRU1NNa2srhw7tx81Ny//+\n73NcuJDNSy/9xdQRzd7x48d55plnUCgUNDU1DSyvqalBq9Xe9PGNjR033eZmzPXTb3PMbY6ZwTxz\nm2NmGHru7l49NQ0dVNV3UFXfTvWl2zUNHfT0DZ78WalQ4O1mg5/WDj+tPf4edvhp7bC30QzeqdFI\nfX3biGU2FXPLfaNi0mwLNICFAf1dtJ0l+5jpfbmLplQouH9+KL//Vwb/3pvHTx6aMu4nvBNCjC5r\nQ1fcsNt1LcPxn8/MmbP5xz9eYc6cZJqaGgkJCQPgwIFU+vr6hrTv8a6mpgZbW1s0mv43Q8HBwZw8\neZL4+Hh27drFww8/bOKEQojRyGg00tLe01+ENVwqxOr7C7H6lq6rttdYKPF0tcHL1RYvF5uB254u\n1lgMQwdemJ5ZF2gOGnuSfGayt+zgVV20cH9npk5w51SujvQLtUyL9DBhUiGEGB2Sk+fx+OOP8dZb\n79PV1cmvfvUsqal7WLfuPvbs2cX27VtvvhNxTTqdDhcXl4H7P/3pT/nZz36GwWBg4sSJzJo1y4Tp\nhBCjQUt7D+W6Nip07VTU9X+trO+gs/vqD8gc7TRE+Dv1F1+uNni52uDlYouzg6XM9zvGmXWBBrAg\nIJmDFUev6qIB3DsvhLMFdXyYWsDkMDf5VEEIMe5FRkZz4MDxgfvvvbd54Pbs2f0jDC5ffs9dzzUW\nxMTEsHHjxoH7oaGh/Otf/zJhIiGEqXR291FZ187pgnouFNb3F2V17bR29A7aTqVUoHW2JjLAGS9X\nGzxdvuyG2WBjZfZv08UdMvuf/OAu2gmSfC9/Qql1tmHBVD++OFHKrvQyls8MNF1QIYQQQggxpvT2\nGahu6LjcFdO1Ua5rv+apie5OVoR4O+KrtcXHzQ4f9/5CTK1SmiC5GM3MvkCDK7toqcz0njaoi7Zi\nViCHM6vYdrSE2bFeONpZmjCpEEIIIYQwV719ejLy6jidq6Osto2ahs5Bc4kBONpqiAp0xsfNjshg\nVxys1Hi72WClGRNvu81OW287BU1FFLeU4ahxINgpAB9bL1TK0Xtm3Zh4pdyoi2ZjpWbNnCD+uSuX\nTw4V8o2lkSZMKoQQQgghzInRaKSwqoW0zGpOZNfQcel6MWtLFcHeDvi42+Lr3j+PmI+77aBRE81t\nZMGxoLm7lfymwkv/iqhsr75qG41KQ6CDP8GOAQQ7BhLk4I+NhbUJ0l7bmCjQoH9Ex+t10ZImebPv\ndAWHzlaRMsUXf4+xMUeCEEIIIYQYGY2t3RzJqiIts5rqhv4pMJzsNCRP9mdWtCfebrYySvgo0NjV\nRF5TIeXF5WRWX6C2o25gnYXSgnDnUEKdggh2DKSxu5mi5mIKmkvIbcwntzEfAAUKvGw9CHIMIMQx\nkCDHANytXU328x0zBZq9xo4k35nsLT3IkcoTJF/RRVMplWxICeVPH5xl0758/t/9k+QXSgghhBBC\nDNLTq+d0no60zGqyixswGkGtUjItUsvsWC+iAl1uOMGzGFlGo5G6zgbymwrJu9Qhq+9qGFhvpbIk\nyjWcMMdgQp2D8bf3Qa0cXO7M9IoHoKO3g8LmEoqaSyhoLqakpYzK9mrSKvsH0rK3sCPYMaC/aHMK\nxM/ed1ADaCSNmQINYKH/XA6WH2VXSSqzvBKwUFkMrIsJdiU22JXMwnrO5NcxOczdhEmFEEIIIcRo\nYDQaKaho4XBmFekXaujs1gMQ4uNAYqwX0yK02FhZ3GQvYiQYjUZqO3TkXnHKYlN388B6G7U1sW5R\nhDkFkxAUg22v4y1fW2ZjYUOMWyQxbv2XP+kNeiraqihoLh4o2s7Wneds3XkA1AoV/g6+BDkGMMsr\nAU/bkZvCa0wVaIO6aFXpg7poABtSQjlf1MAH+/KJDXaVUXOEEEIIIcaphpYujmRVk5ZZRU1jJwDO\n9pakTPFlVownXq62Jk44PvXoe8ltzOd8/QWy6i/Q0NU4sM7OwpbJ7rGEOgUT5hyMl60HSkX/+3l3\nl6Fd76dS9hdg/g6+zPObDfSfPlnQXExhcwmFzcUUt5RR2FxCXWcD34n9+tCe6A2MqQIN+rtoh67T\nRfN2s2Xu5P7r0fadrmBRgp8JkwohhBBCiLupu1fP6VwdaZlV5BQ3YgQ0aiUzoj1IjPEiMsBZTmE0\ngYauxv6CrC6Hi40F9Br654uzVlsxWRtHuHMoYU5BeNho7+plSs5WTsRbTSLeYxIAXX3dlLVWoLVx\nG9HjjrkCrb+LNos9pQeu2UVbNTuIo+dr2Hq4iFkxnthZS8taCCGEEGKsMhqNFFa2cOhcJSdyaunq\n6T+FMczXkcRYLxIitFhbjrm3xKOa3qCnqKV0oCi7cqRFT1sPYlwjiHGNINgxcFQNh2+ltiTMOXjE\njzMmX40L/JM5WH6EncX7ruqi2dtouCcxkE378vn0cBFfWzjBhEmFEEIIIcRIaO3o4WhWNYfOVVFR\n1w6Aq4MlC+L9SIz1xMPZxsQJx5e23nay6y9yvv4C2fUX6ejrP61UrVQT5RJOtFsEMa6RuFm7mDip\n6Y3JAu3KLlpa1Qnm+iYOWj9/qi+pGRWknq4gZYqPnGMshBBCCDEGGAxGzhc3cOhsJRl5degNRtQq\nBQkRWuZM9CIqQEZhvJ6WnlYOVxyjIa8B+pRYqSyxVGmwVFtiqbK84X0rleVVoyUajUYq26vJqssh\nqz6HouZSjPRP6u1k6cgUbRwxbpFMcA7FUqW5VqRxa0wWaHC5i7arOJVEr2mDumhqlZL75oXy0seZ\nfHq4iMdXxZgwqRBCCCGEGApdUyeHz1WRllVFQ0s3AL7utsyJ82amXNJyQ+WtlaSWHeZkTQZ9Rv0d\n70elUGGlskSj0mCltqSzr2tgxEUFCoIc/Ylx7R810dvWU6a8uoExW6DZa+xI9k1kd+n+a3bRJoe5\n4etux8kLOhrmdeHiYGWipEIIIYQQ4nb19uk5nVvHwbOV5JT0j/RnpVGRPMmbpIneBHraj0gRoDfo\nKWktJ8Ded1RdH3U7DEYD5+svsK/s8MBkzVprN+b5zSY5PIEaXTPd+m669T109XVdvq3vpruv59L9\n7kv3r1in76arr5uW7lYUCgXxHpOIdo0gyjUcOws5Y+1W3VKB9vzzz3Pq1Cn6+vr47ne/y6JFiwbW\npaSk4OnpiUrV/wL94x//iIfHyM0LcDvm+ydxoDztml00hULBwgRf3txxgb2nyrl3XqgJkwohhBBC\niFtRWtPKoXNVHDtfTXtXHwATfB2ZM9Gb+HAtlpqRK5p69D28nvUuWfUX8LHz4oHwtQQ5BozY8YZb\nV183x6tPsb/sMLWddQCEO4cyz2820a4RKBVK3GzsMdpIx9GUblqgHTt2jLy8PDZt2kRjYyNr1qwZ\nVKABvPbaa9jajr6qeFAXrfIEc/0Gd9FmRHnw0f4CDpypZGViIFaaMdtQFEIIIYQwW22dvaSeLufg\nuSpKqvvnunK01bB0hj9z4rzxdBn5AT/aezt49dybFDaXoLV2o6KtihdOvUKiz3RWBS/FxsJ6xDPc\nqYauRg6UHyGt8gSdfZ2oFSpmeMWT4jcHHzsvU8cTX3HTiiQhIYG4uDgAHBwc6OzsRK/XD3TMRruB\nLlrJPhK9B3fRLNQq5k72YWtaMWmZ1cyf6mvCpEIIIYQQY5veYKCzW09nd98V//rvd3T30dXT//Wr\n25RUt9LTZ0CpUDAp1I05cV7EhriiVinvSu6m7mZeOrORqvYa4j0m8XDkfRQ1l/Lvix9zuOIYZ3VZ\nrA9dyVSPSaPq2qqi5hL2lR3ijC4Lg9GAvYUdy4IWMsdnBg4ae1PHE9dx0wJNpVJhY9P/qcTmzZtJ\nSkq6qjh79tlnqaioYOrUqTz55JOj6oV5sy7avCm+7DhWwp6TZcyb4oNyFGUXQgghhDA33b16Mgvq\nOXmxlrrmrsvFV7ee7t7bH4RCoQBvNztmRGlJjPXCyc5yBFJfX017LS+e2UhjdxNzfRNZF7YSpUJJ\nmHMwP5n2I/aWHuTz4j28mf0+R6tOsiF8zYhPZHwjeoOeM7pMUssOU9RSCoC3rScpfnOI95g0qFkh\nRqdbPqdvz549bN68mTfeeGPQ8ieeeII5c+bg6OjI9773PXbu3MmSJUuuux9nZxvU6qF339zdb73q\n32C/jIOVR9lTtp9VE+ejueKF6e4Oc6f4sSe9lJK6DqZFeQ45243cTu7RxBxzm2NmMM/c5pgZzDO3\nOWYWQox9Pb16MgvrSb9Qy5n8Onp6DQColAqsLdXYWKpxtNVgY6nG+qp/qoFtBpZpVAO3rTQqtFoH\ndLrWu/68SlrKeOXsG7T1trMyeAmLA+YNakSolWoWB6Yw1WMimy5uIbvhIr8+8SeWBKSwIGDuXc3a\n0dtJWuVxDpQfobG7CYAY10hS/OYwwTlkVDVQxI3dUoF26NAhXn31VTZu3Ii9/eA3B6tXrx64nZSU\nRG5u7g0LtMbGjjuMepm7u/1t/5Imec9kd+l+Pj2796ouWlKsJ3vSS9m8J5cg95G7lu5Oco8G5pjb\nHDODeeY2x8xgnrnNNbMQYmzq7dOTWdjQX5Tl1Q10x7TO1iREaEmI0OKntTPbwiCnIZd/ZL5Dr76X\nByPWkeg9/brbulm78v9NfIwMXSabcz9lW9Eu0msy+O60r+Gh9B6xjH2GPnIbCzijyyK9JoMefQ8a\npQVJPrOY65eIh437iB1bjJybFmitra08//zzvPXWWzg5OV217kc/+hF///vf0Wg0pKens3jx4hEL\nOxTz/ZM4UHGEXSX7mOU9bVAXzVdrR2SAMzkljZTWtOLvIW8ohBBCCCG+qrfPQFZR/UBR1tXTX5S5\nO1mREOFLQoQWfw/zLcq+dKrmDG9nb0KhUPCt2IeZ5H7zOXMVCgVTtHFEuoTxWeFODpYf5Zf7/8I0\nzymsDV2BvcZuWLJ19HaSXX+Bc3XZnK+/QJe+f943J0tHlgUuINF7GjYWIz9oihg5Ny3QduzYQWNj\nIz/60Y8Glk2fPp3w8HAWLlxIUlISGzZswNLSkqioqBt2z0zJXmNHss+sS9eiHWee3+xB6xcl+JFT\n0sjuk2V8c3mUiVIKIYQQQowuvX0Gzhc3kJ5Ty5l8HZ3d/UWZm6MV8yb7kBCpJcBjZOYcM4UD5Uf4\nMPdTLFWWfDfuESY4h9zW463V1tw3YTXTPafyYcEWTlSfJqsuh9Uhy5jpnYBScfsDmzR2NXGuLptz\nuvPkNhVgMPafQupq5cxMrwTi3KMJcQw023nZxGA3LdA2bNjAhg0brrv+kUce4ZFHHhnWUCNlgX8y\nByqOsLsklUTv6YO6aLEhrni42HA8u4b1ySE43uULUIUQQgghRos+vYHsS0XZ6bw6Orv75xtzcbAk\naaI30yI9RmwiaFMxGo1sL9rN58V7sNfY8b2J38LP/s5PTwxw8OM3C57mozO72Fa4k39d/Ihj1ad4\nIHwt3nY3HvPAaDRS2V7NOd15ztWdp7S1YmCdv70PcW7RxLlH423rOaZ+BqLfuJr4y05jy1zfRHaV\npF7VRVMqFCyM9+XdXbmkZlSwek6wCZMKIYQQQtx91Q0dfHG8hFMXdQOTQDvbWzInzouECC3B3g5j\nsiAwGA1syt3C4YpjuFm58P3R1yBUAAAgAElEQVRJ38bdxnXI+1UpVczzm81kbSwf5m7ljC6T36b/\nhfl+SSwNWoClSjOwrd6gp6C5mHN15zmny6a+qwEApUJJhHMYce7RxLlF4WzldL3DiTFiXBVoAPP9\nkthfnnbNLlpijBefHCwkNaOC5TMDsBiG0SaFEEIIIUa7msYOPksr5uj5aoxGcLLTsCDel2kRHgT7\nOIzpaYh6DX28ff59MnSZ+Nh58b2J38LRcnjHI3CydOTbsQ+TVZfDB7lb2F26n1O1Z1kfthKj0ci5\numyy6nJo7+sfTM9KZclU7UTi3KKIco0Y1ZNgi+E37gq0G3XRLDUqkiZ58/mxUo6dr2HOxJEbdUcI\nIYQQwtRqmzrZllbMkaxqDEYjPu62rEoMYkq4+5guyr7U2dfFPzLfIbcxn1CnIB6P+wbW6pErhmLc\nIpngHMKOoj3sLTvIPzLfGVjnZOnIHI+ZxLlFEeYcgoVy3L1NF5eMy5/8fL8kDpSnsesaXbT5U3zZ\ndaKM3SfLmB3nNSbb+EIIIYQY3+qaOtl2tJi0zGr0BiPebrasmh3E1HFSmAG09rTx8tnXKWutYKJb\nNI9GP3hXJnHWqDSsDl3GNM8p7C09iJOlA3Hu0fjZ+9zRACJi7BmXBZqdxpbkS120wxVHSfFPGljn\n4mBFfISW49k1ZJc0Eh3oYsKkQgghhBDDp765iw8OFLL7eAl6gxFPFxtWzQ4iIUKLUjk+CjOAus4G\nXjrzGrrOemZ5TeP+8DV3fQREbztPHo66764eU5iHcVmgQf+8aAfLj/JFyT5meicMamcvSvDjeHYN\nu9PLpEATQgghhNlraOli+7ESDp2tpE9vxMPZmntmBzE90mNcFWYAFW1VvHxmI809rSwOSGFl8GI5\nY0qMKuO2QLOzsGVRwFy2Fn7BrpL9rApZOrAuyMuBUF9HzhXUU1XfjperrQmTCiGEEELcmcbWbnYc\nK+HAmUr69Abcnax4aGkkUX6OqJTj73S6/KYiXj33Jp19XawPu+eqeXGFGA3GbYEGMM9vNgcrjpJa\ndogkn5mDhi1dFO9Hfnkzu0+W8/XF4SZMKYQQQghxe5rbutlxrJT9Zyro7TPg5mjFylmBzIzxxMvT\nEZ2u1dQR76rGribSqzPYUbwbvdHAI1H3M81ziqljCXFN47pA06g0rAhezLs5H7CtaBcPR14+D3jy\nBDdcHaw4klnF2qRg7KxH/qJRIYQQQoihaGnv4fPjJaSerqCnz4CrgyUrZgWSGOuFWjW+OmadfZ1k\n1GaRXn2avKZCjBjRqDQ8HvsI0a7y4bsYvcZ1gQYw3XMK+0oPcrzqFCl+c/Cx8wJApVSyIN6XTfvy\nOXCmguUzA00bVAghhBDiOuqbu9h3upy9p8vp6TXgbG/JhlmBzIkbX4VZn6GP7PqLnKjJILMumz5D\n/2TbIY6BJHhOYYo2DlsLGxOnFOLGxn2BplQoWR26nFfOvs6W/B18b9I3B9bNifNmy+Ei9p2uYPE0\n/3H1B04IIYQQo1tPr57TuToOZ1aRU9yIkf4Jpu+dG0jSRG8s1OPjfYvRaKSwuYQTNafJqDk3MNmz\nh42WaZ5TSPCYhKu1DPomzMe4L9AAolwmEO4cSnbDRS405BHhEgaAjZWaOXFe7DlZzskLtcyI9jRx\nUiGEEEKMZ0ajkcLKFg5nVnEip4bObj0AoT6OzI7zYma0BxbquztcvKlUt9eSXpNBenUG9V0NADho\n7Enxm0OC52T87HxkdEZhlqRAAxQKBatDl/H79L+xJX87/53wxMBEgQvi/dh7spxd6WVMj/KQX3Qh\nhBBC3HWNrd0cPV9NWmYVVfX9HSJne0tSpviSGOuFp8v4OG2vpaeVUzVnOVF9mtLWcqB/TIHpnlNJ\n8JjMBOeQuz6fmRDDTQq0S/ztfUnwmEx6TQYna84MjOyjdbJm8gR3TufqyCtvZoKf0032JIQQQggx\ndL19ejLy6kjLrCarqB6jEdQqJdMitcyO8yIqwGVczGHW1dfNweJs9uYd4UJDHkaMKBVKol0jmOYx\nmVj3aCxVGlPHFGLYSIF2hZXBi8moPcdnhTuZ7B6Lhap/5MaF8b6cztWxO71MCjQhhBBCjBij0Uhx\ndWv/KYzZNbR39Q9yEeztQGKsF9MjtdhYjf2RpTv7Osmsy+FMbSbZDRfpvTTYR6CDPwmek5mqnYi9\nxs7EKYUYGVKgXcHV2oVk30T2lh3kQMURFvgnAzDBz4kAD3tO5+nQNXXi7mRt4qRCCCGEGEua23s4\nmlVNWlYVFbp2ABxtNSyd7s+sWC983GxNnHDktfW2k6nLJkOXyYWGPPTG/uvrvGw9SAyMJ9o+Cq2N\nu4lTCjHypED7isWBKRypSueL4n3M9ErA1sIGhULBogQ/XtuWzd5T5dw/P8zUMYUQQggxBhRXt7D1\ncDHnCuoxGI2oVQriw92ZHedFdJALKuXYHomxpaeVs7rznKnNJLepAIPRAICvnTeTtbFMco/F01aL\nu7v9uJtcW4xfUqB9ha2FDUsCU/gkfzs7S/axNnQFAAmRWj7Yn8/Bs5Wsmh2EtaV864QQQly2detW\nNm7ciFqt5oknniA8PJz//u//Rq/X4+7uzh/+8Ac0GrlORlx2IqeG17fn0NtnIMDTntmxXkyP8sDO\nemyfwtjU3cyZ2izO6DLJbyrCiBGAAAc/Jrv3F2XuNq4mTimE6UiVcQ3JPrPYX5bGgbI0kn1m4Wrt\nglqlZP4UXz4+WMihc1UsSvAzdUwhhBCjRGNjIy+//DIfffQRHR0dvPjii+zcuZMHH3yQpUuX8qc/\n/YnNmzfz4IMPmjqqGAWMRiPbjhTzyaEirDQq/mN9HJNC3Uwda0TVdzZwRpdFRm0mRS0lAChQEOQY\ncKlTFoOLlbOJUwoxOkiBdg0WKgvuCVnC29n/5rPCXXwj+n4A5k72YduRYvacLGPBVN9xMXKSEEKI\nmzt69CgzZ87Ezs4OOzs7nnvuOVJSUvjFL34BwLx583jjjTekQBP09hl46/Mcjp6vwdXBkh+un4iv\ndnQNdlHf2UhLz9BPJzQYDRQ0FZGhO0dpawXQX5RNcAphkjaWie7ROFk6Dvk4Qow1UqBdR7zHJPaV\nHiS95jTz/efgZ++DnbUFs2I82X+mkow8HVPDtaaOKYQQYhQoLy+nq6uLxx9/nJaWFn7wgx/Q2dk5\ncEqjq6srOp3upvtxdrZBPQyTDLu72w95H6ZgjrlvJ3NzWzd/ePMEOcUNhPs78z+PTcPZ3moE013f\n9XIfLTvFX46+PnDa4XBQKZRM9Ixkuu8UEnzicLRyuKP9mOPrA8wztzlmBvPN/VVSoF2HUqFkdehy\nXjzzGp/kb+cHk76NQqFgQbwf+89Usiu9TAo0IYQQA5qamnjppZeorKzk61//Okbj5Te4V96+kcbG\njiHnMNfBFMwx9+1krqhr568fnqWuuYtpkVoeWxZJX1cvuq7eEU55tevlLm+t5OVTb6NRWTDbZwZK\nhj5AiYetlji3KGwt+ifS7mkFXevt/5zN8fUB5pnbHDOD+eW+UTEpBdoNRLiEEeUSTnbDRbIbcol2\nDcfbzZaYYBeyChsoqmohyOvOPgUSQggxdri6ujJ58mTUajX+/v7Y2tqiUqno6urCysqKmpoatFr5\nUG+8Ol/UwCtbsujs7uOexEBWzQ5CoRhdl0m09bTzj8y36TH08p3YrzPRPcbUkYQYt8b22K3DYHXo\nMhQo2JK/fWDo1y8HCNl9ssyU0YQQQowSs2fP5tixYxgMBhobG+no6GDWrFns3LkTgF27djFnzhwT\npxSmkHq6nD9/cJbePj3fXhnF6jnBo6440xv0vH7+Peq7GlkWuECKMyFMTDpoN+Fj58V0z6kcqz7J\n8erTzPSKJzrQBR83W9Jzarl3bijO9pamjimEEMKEPDw8WLx4Mffddx8AzzzzDLGxsTz99NNs2rQJ\nb29vVq9ebeKU4m4yGIz8e18ee06WY29jwQ/WxhHqOzoHxPikYDu5jfnEuUWzNGiBqeMIMe5JgXYL\nVgQv4lTtGbYV7mSqdiIalQULE/x46/ML7DtdzrrkEFNHFEIIYWL3338/999//6Blb775ponSCFPq\n7O7j/7ae51xBPd5utvxwfRzuTtamjnVNx6tOkVp2GE8bLV+P2oBSISdXCWFq8lt4C5ytnJjnN4em\n7mb2lx0GYMaliST3Z1TQ3as3cUIhhBBC3E01HTpydHlXDQBT19zJb989xbmCemKCXPjpQ1NHbXFW\n0lLGvy5+hLXamu/GPYK12jQjSgohBpMC7RYtCpiLrYUNO0tSaetpR2OhYt5kH9q7+jiSVW3qeEII\nIYS4Szr7Ovnzqb/z7L4/8fzJv3G69lz/nF+VzfzqnVOU69pJmeLDD++Nw8ZqdJ6s1Nzdyj8y30Fv\n0PNo9INobdxNHUkIcYkUaLfIWm3N0sAFdOm7+KJ4LwApU3xQKRXsTi/DcItDKAshhBDCvG0v2k1r\nbxsBjj6UtVbyeta7/M/B3/OHnVtp7eziwQVhPLQoHJVydL7N6tP3sTHrnzR1N3NPyBKiXcNNHUkI\ncYXR+ZdjlJrjMwM3KxcOVhxF11GPo50l06M8qG7oIKuw3tTxhBBCCDHCKtuqOVB+BDdrV36z8Gme\nmf4kPsoImnubUAVk4jr9KEqPYrr1PaaOel1vZHxAYXMxU7UTWeg/19RxhBBfIQXabVAr1dwTsgS9\nUc9nhV8Al4fc/+J4qSmjCSGEEGKEGY1GPszbisFoYH3YSowGJVv36cg/Foh14SISXKfTa+zio7zP\n+N8jv2FH0W7ae4c++fhwOlRxjD0Fh/Cx8+JrkfeOuiH/hRAyiuNtm6yNI6D0EKdqz5LSModAD/+B\niavPFzUQHeRi6ohCCCGEGAEZukxyG/OJdo0gwCaUZ149Qk5xAyHeDnx/XRyOthraepawvzyNA+Vp\nbC/azZ7SA8z2nkGK/xycLE07zH5BUzEf5n6KvaUd3419BEuVxqR5hBDXJh2026RUKFkTugyALfk7\nMBqNrEvqH2Z/84ECuRZNCCGEGIN69D18nLcNlUKFb+80frbxODnFDUyL1PLUA5NxtO0vduw0tqwI\nXsRzs37CmtDlWKms2Ft2kGeP/I5/XdhMbUedSfI3djXxWtY7GDHynzO/hau1fKAsxGglBdodCHMO\nIdYtkrymQrLqcwjwtGdapJaS6lZOXdSZOp4QQggxpjR3t/CnU3/nTG2myTLsKkmlsbsJi4ZQtuyp\npbvXwKMrovjuPdFoLFRXbW+ltmKBfzK/mPVjHgxfh7OVE2mVJ/jlsT/wRtZ7lLVW3rXsvfpe/pH5\nDq09bawNXUGMhwwKIsRoJqc43qFVIcvIqrvAlvwdRLmEsyYpmFMXdXx8oIDJYW6oVVL7CiGEEMPh\ni+K9FDQXUdZWgaetB5622rt6/MzyMr4oSsXYa0lToR9JE71YPSeYsCA3dLrWGz7WQqkm0Wc6M70T\nyKg9x86SVE7VnuVU7VmiXMNZHJBCqFPQiGU3Go28f/FjSlvLmeEZz1zfxBE7lhBieEiBdoe8bD2Y\n5Z1AWuUJjlWdJNFnOkkTvUnNqCAts4rkST6mjiiEEEKYvYauRo5UnsBabU1nXyevZ73LU/E/QKOy\nGPFjN7d188mhIo51bEPlbEDbOYVvPjILP63dbe9LqVAy1WMSU7QTyW64yM7iVLLrL5Jdf5FgxwBm\ne89gsjYWzTBfF7a/PI3j1acIcPDj/vA1MiiIEGZACrQhWBa0kPTqDLYV7SLeczIrEwNJy6zi08NF\nzIz2vOYpD0IIIYS4dTuL99Fn1PNA2EqKW8o4VHGUzXmf8mDE+hE7Zk+vnp3pZew4VkKvdTWW4bV4\nWfry01WrUA5xbjOFQkG0awTRrhEUNBWzq2QfWfUXKGwu4cO8T0nwmMws7+n42XsP+XlcaMjj4/xt\nOGjs+U7s17G4C0WtEGLopEAbAidLR+b7J/F58V72lB5gedBCFib4sf1oCXtPlbN0RoCpIwohhBBm\nq76zgSNV6Wit3UjwmMxU7USKmktIqzxBmFMICZ6Th/V4BqOR4+dr2HyggMbWbuxsVThEFtBuUPCN\nuPVDLs6+KsQpkP9weoy6znqOVKZzrCqdgxVHOVhxFH97H2Z5TyPeYxLWauvb3nddZwNvnH8PBQq+\nHfuwyUeQFELcOrlQaogW+CfjoLHni+K95DcVsXS6P7ZWarYfLaG9q9fU8YQQQgiz9UXxXgxGA0uD\nFqBSqrBQWfDNmK9hqdLw/sWPqOkYvoG5csua+NXbJ3ltWzatHb0smxHAkmV62gxNzPGZie8wdLSu\nx83alXtClvDcrJ/yeNw3iHWLorytin9f/ISfHP4V72RvIr+pCOMtjhTdre/hH5lv097bwYYJqwl2\nDByx7EKI4ScF2hBZqa14LPpBADZm/pMeRQfLZgbQ0d0nk1cLIYQQd0jXUc+x6lN42GiJ95g0sFxr\n486D4evo1vfweta79OqH9mFoTWMHL3+cye/eO01xdSvTozz4zbens2CmG3vK92FrYcOK4EVDfTq3\nRKVUEesWxeNx3+C5WT/hnuAlOGrsOV59ij+f/jvPHX+BPaUHaO1pu+4+jEYj/8z5gIq2Kub4zCTR\nZ/pdyS6EGD5yiuMwCHMOYW3oCjbnbeW1zH/yvUnfZnd6GbvTy5g/1RcnO0tTRxRCCCHMypfds2VB\nC1AqBn+eHO85mdymQtIqj7M5/zMeCF972/tv7+rls7Ri9p4qR28wEurjyIb5oYR4958K+Nb5T+nW\n97A2dAW2FjbD8pxuh5OlI4sDU1gYMJe8xkKOVJ3gTG0mn+RvZ2vBF8S5RTHLexoRLmGDvj+7S/aT\nUXuOEMdA1oetvOu5hRBDJwXaMJnrm0hJSxnpNRlsKdrGqtkzePuLi3yWVszDi2W+ESGEEOJW1Xbo\nOF59Ci9bD6Zo4665zfqweyhuKeVwxTEmOAUz9You24306Q2knq5ga1oR7V19uDlacd+8UKaGuw+M\ncFjQVEx6zWn8Ll0HZkpKhZJwl1DCXUJpm9BOenUGRypPkKHLJEOXiYuVMzO94pnplUBFWxVbC7/A\nydKRb8U+jFopb/OEMEfymztMFAoFD0aso6q9hrTK49w/wQcPFxsOnq1k0TQ/PJzv/qdvQgghhDna\nUbQXI0aWBS28qnv2JY3Kgm9Gf43fnfwb/7rwEX72Pmht3G+437qmTv6y+RyVde1YW6q5b14o86f6\nYqG+fAyD0cAHuVsAuG/C6use3xTsLGyZ5zebub6JFLeUcaTyBCdrz7C9aDc7ivagVqpQK1V8N/YR\nHDT2po4rhLhDo+evzhigUWn4duzXsVXbsDnvU+bMsEZvMPLJwUJTRxNCCCHMQnV7LSdrMvCx82KS\ne8wNt/Ww1fJA+Fq69N28nvXeDa9HK6tt49fvnqKyrp3kSd787rszWDLdf1BxBpBWeZzytkqme04l\n2HF0jsasUCgIcvTna5Hr+W3iM3wtYj2BDn7ojQYejFiPv4OvqSMKIYZACrRh5mbtwqPRD6I3Gkhr\n/Qw/bwtO5NRSUt1q6mhCCCHEqPd58Z6bds+uNM1zCrO8plHeVsnH+duuuU1uWRO/e+80zW093D8/\njEeWRGBvc/WE0G297XxWsBMrlSWrQpYN+bncDVZqK2Z5T+P/xX+fPyU9xzTPKaaOJIQYIinQRkCk\n6wTuCV5CU3cLmtAzoDDw0cECU8cSQgghRrXKtmpO1ZzFz86biW7Rt/y4eyeswtvWk4MVRzlVc3bQ\nuow8HS9sOkNPr57vrIxiUYLfdfezrXAX7X0dLA1agKOl+Z0iKBNRCzE2SIE2QhYGzGWSeyyVXWVo\no4rJKmwgM7/O1LGEEEKIUWvHpe7Z8uBFAwN23AqNyoJvxjyERqXhXxc2U9vR///tobOVvPRxJgoF\nPLE+jhnRntfdR1lrJYcrjuFho2Wub+KQn4sQQtwpKdBGiEKh4OHIe/G09aDVNheVayVvb8++5Ukm\nhRBCiPGktKmCjNpz+Nv7EuMaeduP97zierQ3st5j65EC3vz8ArZWFjz1wGRig12v+1ij0ciHuVsw\nYuTeCffI6IdCCJOSAm0EWamt+E7s17FSWWEZfJ7cuhIy8qSLJoQQQnzVB+f7rx9bHrTwtrpnV5rm\nOYUZXvGUtVWwveRzXBws+clDUwbmNrue9JoMCpqLmegeQ6TLhDs6thBCDJdbKtCef/55NmzYwLp1\n69i1a9egdUeOHGH9+vVs2LCBl19+eURCmjMPG3ceidqAUaHHMiyDDw9lYzBIF00IIYT4UllrBSfK\nzxDo4E+0a8Qd76dPb6A9LxxDhx1qj1LuWWaFl6vtDR/T1dfFlvztWCjVrAtdccfHFkKI4XLTAu3Y\nsWPk5eWxadMmNm7cyG9+85tB63/1q1/x4osv8v7775OWlkZ+fv6IhTVXce7RLA1cgMKyk0aXYxzO\nrDR1JCGEEGLU2F60G4AVQbd37dmVunr6+Nvmc5zIrsezdQ4WSgs+LfmUus76Gz7ui+J9NPe0stB/\nLq7WLnd0bCGEGE43LdASEhL461//CoCDgwOdnZ3o9XoAysrKcHR0xMvLC6VSSXJyMkePHh3ZxGZq\nWdACYtyjUDnW89HFHfT26U0dSQghhDC50pZyMuuyCXcLIcIl7I720drRwx/eP0NWUQNxIa78eH0S\n94evoUvf1T8/mqHvmo+raa9lX9khXKycWRgwbyhPQwghhs1NCzSVSoWNjQ0AmzdvJikpCZVKBYBO\np8PF5fKnTS4uLuh0uhGKat6UCiVPzv4m1jjQ55bHeycOmjqSEEIIYXLbi/ovndgQs+KOumd1zZ38\n9t3TFFW1kBjjyffXxmJpoWKGVzwzPOMpbS1nS/72qx5nNBrZnPcZeqOedaEr0MgQ9UKIUeKWhyna\ns2cPmzdv5o033hjSAZ2dbVCrVUPaB4C7u/nNTwLw46T/4NnUP5LevovVxkmEaf1NHemWmOP32xwz\ng3nmNsfMYJ65zTGzENdT1FxKVv0FQp2CiNaGU1fXdluPr9C18acPztLY2s2S6f7cOzdkUJF3X/hq\nilvL2F+eRphzCJPcYwbWZdZlk91wkQjnMCZesVwIIUztlgq0Q4cO8eqrr7Jx40bs7S+/OdBqtdTV\nXR6VsKamBq1We8N9NTZ23GHUy9zd7dHpWoe8n7vN3d0eN7UrE63mc7Z3F79OfYVfJv0X1mprU0e7\nIXP8fptjZjDP3OaYGcwzt7lmFuJ6vuye3cm1Z/nlzfx181nau/q4b14oS6Zf/YGnpUrDN6O/xvMn\nX+TdnA/wtfPGzdqFXn0vH+V9hlKh5N4J99zxdW9CCDESbnqKY2trK88//zz/93//h5OT06B1vr6+\ntLW1UV5eTl9fH6mpqSQmyuSON/PI9Hko60LooJnXz72PwWgwdSQhhBDiripsLianIZcJzqGEOYfc\n1mPP5tfxx39n0Nmt55vLI69ZnH3J286TDRNW09nXxRtZ79Fn6GNP6UHquhqY65uIp63HUJ+KEEIM\nq5t20Hbs2EFjYyM/+tGPBpZNnz6d8PBwFi5cyM9//nOefPJJAJYtW0ZQUNDIpR0jLDUqVoUsZXPp\n++RwgZ3F+1gatMDUsYQQQoi7Znth/8iNy4MW3tbj0jKreHPHBdQqBU+sjyUuxO2mj5nhFU9eUyHH\nq0/xbs6HnNFlYa+xY9ltHlsIIe6GmxZoGzZsYMOGDdddn5CQwKZNm4Y11Hgwd5Ivu05No80qle1F\nu/Gz9yHGLdLUsYQQQogRl9dYyIXGPCKcwwh1uvUPdj8/XsKHqQXYWqn54b0TCfW58QTUX1IoFNw3\nYTXFLWWk12QAcH/IGqzVVneUXwghRtItTVQthp9apWRtYiTdeZPBqOCt7Pep7ai7+QOFEEIIM/fl\ntWfLgxfd0vYGg5FN+/L4MLUAZ3tLfvy1KbdcnH3JSm3Jt2IeQqPSEOIYxDTPKbedWwgh7gYp0Exo\nWpQHvnY+dBdF09nXxWuZ79DV123qWEIIIcSIyW3MJ6+pkCjXcIIdA266fU1DB7977zQ7T5Th5WrD\nTx+aio+73R0d29vOk+dm/oQnJn8bpULeAgkhRif562RCSoWCdckh6Ot8cOqaQGV7Ne/mfCCDhggh\nhBiTjEYj2y5de7Yi6MbdM4PRyJ6TZTz7xgnyK5qJj9Dyk4em4uo4tNMS7TS2qJW3PMuQEELcdfIX\nysRig12Y4OdEbmYgoUmdZOgyeTv733w9cgMq5dDnixNCCCFGi4uN+RQ0FxHjGkmAg991t6tr6uSN\nHTlcKG3CztqCx5ZHMi1SRlsUQowPUqCZmEKhYH1yCL95twljYTzB4UpO1pxBbzTwaNQDUqQJIYQY\nE/q7Z19ee3bt0RONRiMHzlayaV8+3T16JoW68ciScBztLO9mVCGEMCkp0EaBUF9HJoW6cSa/jv9I\nWI1SuZWM2nMYDHoei/manIohhBDC7GU35FLUUsJEt2j87X2vWl/X1MmfPzhLVlED1pZqvrk8klkx\nnjKJtBBi3JF3/qPE2uRgzubXsfVQOU8/9CivZ7/D2brzvJb5Dt+KeRgLlYWpIwohhLiO48eP88Mf\n/pCwsDAAJkyYwP/P3p2HR1Xe7x9/z5JJMtlnMtnJHiAQ9kVAAQFBcMWlLpRqq7Zat9baovZXq639\ntrVaq622Vqq0dam0aK1bhYIICCGyQ9iSsGbf94Ukk/n9EYgiyJZlZsj9uq5cTObMOXNnriFnPjnP\n83nuuOMOFixYgNPpxOFw8NRTT2GxWNyc1D1cLldX58Yvrz3mcrlYl13CmytyaWxpJyPJxjfnDMYW\nrBb4ItI/qUmIh4hzBDJlZAyF5Y28s+oQ3x3+LdJtA8mu3MOfd/yNVmebuyOKiMgpjB8/nldffZVX\nX32VRx99lN///vfMmzePN954g4SEBJYsWeLuiG6zs3IPh+ryGekYRlxQTNf9tQ1H+MNbO3j5g910\nuFzcOnsQD9wwQsWZiFSEi/EAACAASURBVPRrKtA8yE3T04gND+DjzYVszanizmG3kmEfzO6qHP60\nfRFHnK3ujigiImcoKyuLGTNmADBt2jQyMzPdnOjMrSnM5LF1v+b5rX9hSe67rC3MIq/mAI1tTWd9\nLJfLxfsHlmHAwOVfuHr22e5SfvKXLLbmVTA4PpQ//HA6U0fGakijiPR7GuLoQXwtJu6+JoOf/20j\ni/67hwERY/n2sFt4Jft1tlXs5I/bXua7w7+Fn1l/WRQR8TR5eXncdddd1NbWcu+999Lc3Nw1pNFu\nt1NeXu7mhGemraOdD/b/j4a2RipaqthdlXPc9iBLINHWSKICIokKiCA6IIKogEiCfAJPWlxtr9hF\nfn0hYyJGEBMYRV1TK68ty2HjnjIsPka+PnMg00bHEmmzUl5e31c/poiIx1KB5mGi7QHcOnsQL727\niz++k81PbhnL7RnzWbTrH2wp284L217m7hG3468iTUTEYyQmJnLvvfcyZ84c8vPzueWWW3A6nV3b\nXS7XGR0nLMyK2dz97r0OR9A57/vpoc+ob2vgioEzuH7o5RTWl1BQW0xBXTGFdSUU1BWTU7OPnJp9\nx+0XaAkgNjiKuOBo4oKjiAuJJjY4imWbV2DAwNfHXE3eoXr+uGQ7NQ1HSE+08f2bRxET/vmi093J\n7S7emBm8M7c3ZgbvzO2NmcF7c3+ZCjQPNGFIFLn5tazcUsjry3K47fL0zpb7hs4W/H/YupB7R9yB\n1cff3VFFRASIjIzksssuAyA+Pp7w8HB27NhBS0sLfn5+lJaWEhERcdrjVFef/RDCL3M4grp1Jer9\n3R8DMMY2hsbadkIJJzQonIygYRDb+ZgjzlZKm8ooaTz2VUpxUym5lQfYW7HvhGOODB/B35ccInNn\nKWaTkRumpTJr3ACMLldX1u7mdgdvzAzemdsbM4N35vbGzOB9uU9VTKpA81A3zUhjf3Edn+4oJm1A\nCJOHx3DrkJswGUxklWzi91tf4r6R3ybAx+ruqCIi/d67775LeXk5t99+O+Xl5VRWVnLttdeydOlS\nrr76apYtW8bkyZPdHfO08uuL2F97iCG2QURYw7/ycb4mC/FBcSe0y2/raKe8qYLixlJKmjoLt5K6\nWnZlhlNbXUpSdBC3Xz6EmPCA3v5RRES8lgo0D+VjNnL33AweX7SB15blkBgVzICIQOanfw2TwcS6\n4s94bsufuW/ktwmyBJ7+gCIi0mumT5/OD3/4Q1asWEFbWxuPP/446enpPPTQQyxevJiYmBjmzp3r\n7pintbpgHQBT4iae0/4+RjMxgVHEBEZ1Hm9bEWtX78FkNHDtlCTmTIjHZFR/MhGRU1GB5sEcof7c\ncXk6f3h7B3/89w5++s1x+PuauXnwtZiMJtYUZvLclj9z/6jvEGw5P8bcioh4o8DAQF588cUT7l+0\naJEb0pybprZmNpRuweYXxlD74G4fr7SqiTeW5xDgZ+ZHN48iPlLnKRGRM6E/Y3m4UQMdzB4fT2l1\nM3/97x5cLhdGg5EbB85lWtxFFDeW8uzmP1NzpNbdUUVExIutL9lIW0cbk2MnYDR07+OBs6ODv7y/\ni9a2DubPGqTiTETkLKhA8wLXTk0mNS6EDXvK+HhzIQAGg4Hr0q5kRvwUSpvKeHbzi1S31Lg5qYiI\neKMOVwdrCjIxG81Mih7f7eN9uP4w+4rqGJ8ewQVDInsgoYhI/6ECzQuYTUbuumoogf4+vLkilwPF\ndUBnkXZNyuXMTphOeXMlv9v8IpXNVW5OKyIi3mZvVR5lzRWMiRhBoKV7DTwOldTz7qcHCA20MH/W\noB5KKCLSf6hA8xK2YD/uvGooHR0u/vjvbBqa24DOIu3KlNlcnjSTypYqfrf5RcqbKt2cVkREvMmq\nwu41Bzmmrd3Jwvd34exwcdvl6QT6+/REPBGRfkUFmhcZmmTjygsTqaxr4eX3d9HxhYVPL0uayVXJ\ns6k+UsOzW16ktKncjUlFRMRbVDZXk12xm/igWBKCBnTrWG+t2k9RRSPTR8eSkWTvoYQiIv2LCjQv\nc9WFSQxJDGPbvkqWZh0+btulidO5JvVyao7U8uzmF8mtPnGxUBERkS/6tGg9LlxMiZ2EwWA45+Ps\nOVTN/zbkE2mz8rVpqT2YUESkf1GB5mWMRgPfuXIooYEW3lq1n72Hq4/bfkn8VG4YOJeGtkae2/IS\n7+9firPD6aa0IiLiydo62llX9BkBZitjIkee83GaWtp5+YNdGAwGvn3FEHx9TD2YUkSkf1GB5oWC\nAyzcdXUGAC++u5Paxtbjtk+Nm8QDo7+LzS+U/x5cwbNb1DxEREROtKVsOw1tjUyIGYvFdO7zxf6x\nPIfKuiNcMSmB5JjgHkwoItL/qEDzUgMHhHLd1GRqG1p56d2ddHS4jtueHJLAI+O/z5iIEeyvPcSv\nNjzLptKtbkorIiKeaHXBOgwYmBJ77s1BNu0tZ212CYlRQVwxKbHnwomI9FMq0LzYpRfEMzI1nN2H\nqnl37YETtvub/fnW0HnMT78Bp6uDV3a+wau7/0lL+xE3pBUREU9yuL6AA3WHGWIfRLj/uTX0qG04\nwt8+2oOP2ci3rxyC2aSPFSIi3aXfpF7MaDBw+xXphIf48d7ag2QfOLG9vsFgYGL0WB4e9z0GBMWy\nvngjT258jsP1BW5ILCIinmJNQSbAOV89c7lc/PW/e2hobuP6i1OItndv/TQREemkAs3LBfj58N25\nGZhMBl56dxdVdS0nfVyk1cEPx9zDjAFTKGuq4OmNL7Di8Go6XB19nFhERNytqa2JDaVbsfvZGGI/\nt8Wk12wvZtu+StITwpgxJq6HE4qI9F8q0M4DSdHB3Dg9jYbmNl78z07anScvusxGM9emXcE9I27H\n6uPP23nv88dtr1DXWt/HiUVExJ0yizfS1tHG5NgJGA1n/1GgrKaZf6zIxd/XzO2Xp2PsRnt+ERE5\nngq088T00bGMT48gr7CWt1adev2zIfZB/L/xP2CIfRC7q3L4Zdbv2Fm5t4+SioiIO3W4OlhdmImP\n0czEmHFnv3+Hi5ff38WRVifzZw3EFuzXCylFRPovFWjnCYPBwK2zBxNps7L0s3y25JSf8vFBlkC+\nO/xbXJd2Jc3tzfxx28u8lfsebR3tfZRYRETcYU9VLhXNlYyJGEmgz9nPG1v62WFyC2oZOziCCUMi\neyGhiEj/pgLtPOLva+aeuRlYzEb+8sFuiioaT/l4o8HI9AGT+eHYe4m0Ovg4fw2/3fg8pY1lfZRY\nRET62urCdQBMiTv75iD5ZQ38e81+QgIt3HLpIAwa2igi0uNUoJ1n4iICuXX2YJqPtPPbxVuprD15\n05AvGhAUy0Pjvsek6PHkNxTx6w3Psa7oM1wu12n3FRER71HZXEV2xR4SggaQEDzgrPZta+9g4Xs7\naXe6+NacdAL9z31haxER+Woq0M5DEzOi+Nq0FKrrj/DMP7dS39R62n18TRa+nn49t2fMx2Q08fqe\nJby883Wa2pr7ILGIiPSFNYXrceE6p6tn76zZT0F5IxePjGF4yrmtmyYiIqenAu08NeeCBGaPj6e4\nsoln/7WdltYzm1s2OmI4j4x7gOSQRLaUbeeXn/2OTw99hrPD2cuJRUSkN7U521hX/BkBPlbGRIw4\nq31z8mv4KOswEaH+3DA9tZcSiogIqEA7r10/LYULM6I4UFzHC//O/sr2+19m9w/j+6Pu5LLES6ht\nreP36xfx86ynWVuYpSYiIiJeanPZdhrbmpgUPR4f05kPT2w+0s5f3t8FBrjjyiH4Wcy9mFJERFSg\nnceMBgO3zhnMiBQ7Ow9U8Zf3d9FxhvPKTEYTlyfP4rEJP+KSlMnUtNTwxt63eDzzST7OX8MR5+mH\nTYqIiOdYXZiJAQMXxU44q/3eXJFLRW0Ll01IIDU2pJfSiYjIMSrQznNmk5G75maQFhfCZ7vL+Mf/\ncs+q+Ue4v53vjJ3HzyY9zPQBk2lqa+Kt3Pf46bpf8dHBFZqjJiLiBQ7XFXCw7jBD7YMI97ed8X5b\ncstZs72Y+MhArr4oqRcTiojIMSrQ+gFfHxP3Xz+cOEcAKzYX8N7ag2d9jFDfEK5Lu5InJv2YOYkz\ncLo6eG//Uh5d9yv+s++/1Lc29HxwERHpEau6WutPOuN96ppa+dt/92A2Gfn2FUMwm/SRQUSkL+i3\nbT8R4OfDAzeMJDzEj3c+PcDKzQXndJxASwBXJF/KE5Me4eqUOfgYzSw7tJJH1/2Kf+X8h+qWmh5O\nLiIi3dHQ1sim0q2E+9tJtw08o31cLhd//2gvdU1tXDc1mVhHYC+nFBGRY1Sg9SNhQb48eONIgq0+\nvLYshw17zn1Ban+zH7MSpvHzSY9ww8C5BPoE8EnBWh7LfJLXd/+LsqbyHkwuIiLnan3xRto62pkc\nOwGj4cxO++t3lbI5p5zB8aHMHHd266WJiEj3qEDrZyJtVh64YSS+FhMvvbuTnQerunU8i8mHqXGT\neHziAuan34DdP4x1xRv4+fqneSX7dQobinsouYiInK0OVwdrCtfjYzQzMXrcGe+XtasUgG9elo7R\nYOiteCIichIq0PqhhKgg7rtuOAYDPP/WDg4U13X7mGajmYnRY3n0gh9ye8Z8YgOj2VS2jV9+9jv+\ntG0RB2oP9UByERE5G7urcqhormRM5EgCfKxnvF9heQOhgRYiQv17MZ2IiJyMCrR+Kj0hjDuvGkpr\nu5Pf/XMbxZWNPXJco8HI6IjhPDzue3x3+LdIDkkgu3I3T296gee2vEROdd5ZdZEUEZFzt7qgsznI\n1Ngzbw7SfKSdyrojmncmIuImWm2yHxszKIJbLh3E3z7ayzOLt/LI/DHYgv165NgGg4GM8HSG2geT\nV3OAjw6uYE91LjnVeSQFJzA7cTpD7YMxaOiMiEivqGiuYmflXhKD44kPjjvj/QorOv9gFxse0FvR\nRETkFFSg9XNTR8ZS39TG26v388w/t/Hw10cT6O/TY8c3GAykhSWTFpbMobp8Pjr4MdsrdvKn7YsY\nEBjD7MQZDHcMPeOJ6yIicmY+LVyPCxdTYiee1X6F5Z3LpqhAExFxD30qFi6fmMDMsQMoqmjkuX9t\n40irs1eeJyF4AHcOv5Ufj3+AMREjKGgoZmH2q/zfZ7/js5LNODt653lFRPqbNmcb64o/I9AngNER\nw89q38Lyo1fQNMRRRMQtVKAJBoOBG2ekMmFoJPuK6njhnR20Ozt67fliA6O5LePrPHrBg1wQNYay\npnL+tutNfp71NOuKPqO9o73XnltEpD/YVLaNxrYmJkaPw8d0dqMijg1xjAk/86YiIiLSc1SgCQBG\ng4HbLktnWLKd7P1VvPLhbjp6uZlHZEAEtwy5kccmLOCi2AnUtNTw+p4lPJ75Gz4pWEurs61Xn19E\n5Hy1uiATAwYmx044630LKxoJD/HDz6JZECIi7qACTbqYTUbunptBSmww63eW8uaK3D7puBjub+Pm\nQdfys0kPM33AZBraGvlXzn/4aeav+N+hT2hpb+n1DCIi54u8yoMcqs8nI3wwdn/bWe1b19RKXWMr\ncRreKCLiNmdUoOXk5HDJJZfw2muvnbBt+vTpzJs3j2984xt84xvfoLS0tMdDSt/xtZj43vUjiAkP\nYPnGAj7I7Lv1y0J9Q7gu7UqemPQIsxKm0eZs4519H/LTdb/mvweW09TW3GdZRES81bK81QBMOYvW\n+scUlR8b3qgGISIi7nLa8QtNTU088cQTTJz41V2gFi5cSECAfpmfLwL9ffjBDSP41WubeHv1fgID\nfZk6LKrPWuIHWQK5OmUOM+OnsqpgHSvzP+X9A8tYfng1U+ImMjF6LOH+dnV+FBH5koa2Rtbmb8Th\nb2ewLe2s9+9qse/QOV1ExF1OW6BZLBYWLlzIwoUL+yKPeAhbsB8P3jSKp/6xhb9/uJtDRbXMnzUQ\nk7HviiKrj5U5SZcwbcBFrClcz4r81Sw7tJJlh1biY/QhyuogKiCKmIBIogIiiA6Iwu4fpsJNRPqt\nzKINtDnbmBw78Zx+F6rFvoiI+522QDObzZjNp37YY489RmFhIWPGjOHBBx/U4sPniSiblZ/cMpYX\n/p3Nqq1FVNa18N2rM/D37duJ435mP2YmXMzUuAvJKtnEvpoDFDeWUtJURn5D0XGP9TH6EBUQQaIt\nFpvJTnRgJNEBkdj8VLiJyPkvu3I3FpMPE6PHntP+hRWNGA0Gou3q4Cgi4i7d/qR9//33M3nyZEJC\nQrjnnntYunQps2fP/srHh4VZMZtN3X1aHI6gbh/DHbwtt8MRxK/uuZAnX93I5j1l/HbxNn56xwXY\nQ/zdkufaqJldtzs6OihrrCC/rpj82iIKaospqCumsK6E/PrC4/azmHyIDY5iQHAMcSHRxAVHk2JL\nIMw/pK9/hNPytvcIeGdm8M7c3phZ+s61qVcQGOyLlbMvsFwuF4XljUTa/PHpgfO0iIicm24XaHPn\nzu26PWXKFHJyck5ZoFVXN3X3KXE4gigvr+/2cfqaN+f+7lVDeM3PzKqtRTzwu1U88LURxEW4v8uX\nCX8SLckkOpLB0Xlfh6uDDv8j7MzfR3FjaddXfm0xB6rzu/Y1YGBQWCoXRI9hhCMDX5PFTT/F57zx\nPeKNmcE7c3tr5v6kpaWFK664grvvvpuJEyeyYMECnE4nDoeDp556Could3/PJAQPOOf3SU1DK01H\n2hmSGNYLyURE5Ex1q0Crr6/n+9//Pn/605+wWCxs2LCBSy+9tKeyiQcxGY3ccukgwkP8eGvVfn71\n+ibuvmYYQxPProVzXzAajEQGRWB2+DPCkdF1v7PDSUVLFSWNpRQ1lLKrag97qnPZU52Lr8nCKMdw\nLogeQ2pokoZDisg5+dOf/kRISOeV+d///vfMmzePOXPm8Mwzz7BkyRLmzZvn5oRf7dj8M3VwFBFx\nr9MWaNnZ2Tz55JMUFhZiNptZunQp06dPJy4ujpkzZzJlyhRuvPFGfH19GTJkyCmvnol3MxgMXD4x\nEXuIH698sJtn/7mNW2cP5qLh0e6OdkZMRhORVgeRVgcjHBnMSZpBWVM5WSWb+axkM+tLNrK+ZCM2\nvzDGR43mgqjRRFgd7o4tIl5i37595OXlcfHFFwOQlZXFz372MwCmTZvGK6+84tEFWsHRFvtaA01E\nxL1OW6BlZGTw6quvfuX2W2+9lVtvvbVHQ4lnmzAkirBAX55/ewevfLibitpmrr4oySubw0RYHVyZ\nfCmXJ80kr+YAWcWb2FK+nY8OruCjgytICk7ggugxjIkYgdXHPfPuRMQ7PPnkkzz66KO88847ADQ3\nN3cNabTb7ZSXl5/2GO6cp13ZcASAjIERbhua6o1DYr0xM3hnbm/MDN6Z2xszg/fm/rK+bccn541B\n8WH8+Btj+N0/t/Hu2oNU1LbwzTmDMZu8c2ig0WBkYFgKA8NSuME5l23l2WQVb2JvdR4H6g6xJPdd\nhoUPYULUGNJtAzEZNYFeRD73zjvvMHLkSAYMGHDS7S6X64yO48552vsLajCbDPjQ4Za5jt46x9Lb\nMoN35vbGzOCdub0xM3hf7lMVkyrQ5JxF2wP4f7eM5fdLtrEuu4Tq+iPcc00GVj8fd0frFl+ThfFR\noxkfNZrqlho2lGxhfckmtpRtZ0vZdoIsgYyLHMUFUWOIC4pxd1wR8QCffPIJ+fn5fPLJJ5SUlGCx\nWLBarbS0tODn50dpaSkRERHujvmVOlwuCisaibYH9Ol6lyIiciIVaNItIQEWFtw8mpfe28mW3Ap+\n9dpmvve14YS7qQ1/TwvzC2VW4jRmJlzM4foC1hdvYlPpVj7OX8PH+WuIDYxmfNRoMuyDibRGeOUw\nTxHpvmeffbbr9h/+8AdiY2PZsmULS5cu5eqrr2bZsmVMnjzZjQlPraK2hda2DmIdahAiIuJuKtCk\n23wtJu65Zhhvrshl+aYC/u/vm/j+10aQEHV+jAOGzgYpCcEDSAgewHVpV5BduYes4k1kV+7m33kf\n8O+8DwjzDSXdlsZg20AG29II8NFCryL92X333cdDDz3E4sWLiYmJOW5ZGk9zrINjrDo4ioi4nQo0\n6RFGo4F5MwcSHurP4hW5/Pr1zdx19VBGpIa7O1qPMxvNjHRkMNKRQUNrI9srdrGnKoc9VbmsK97A\nuuINGDAQHxzHENtABtsGkhQcr3lrIv3Efffd13V70aJFbkxy5gqPdnCMDVcHRxERd1OBJj1q1rgB\n2IN9eem9Xfz+re3MnzmQaaPj3B2r1wRaApgUM45JMePocHWQX1/I7qocdlXmcKDuEIfq8vnvwRX4\nmfwYFJbCYNtA0m0DcVjt7o4uItKlsOJogaYhjiIibqcCTXrcmEERLAj05bkl23l1WQ7ltS1cf3EK\nxvN8fpbRYOwaBjk7cQbN7S3kVu9jd1UOu6ty2Faxk20VOwEI97eTfrRYGxiWgr/Zz83pRaQ/Kyxv\nwNfHhD1Ev4tERNxNBZr0ipTYEH5yS2cb/o+yDlNZ28IdV6Tj0wPr+3gLf7Mfwx1DGe4YCkBFc+XR\nYi2XvVV5rCnMZE1hJkaDkaTgeAaFpRJXF0Fbkwtfsy9+Jl98zb74mo7eNvnia7KoEYmI9Kh2Zwcl\nVU0MiAg67/+QJiLiDVSgSa+JCLN2tuF/azsb9pRxuLSeG6enMSLV3i+LjHB/O5NjJzI5diLODicH\n6/LZXZXDnqoc9tceYl/tQTh46mMYMOBrsnQWbV8s3szH/xvmG4rDaifc347dLwyzUf/VReTkyqqb\naXe6NLxRRMRD6FOb9KpAfx9+dNNI/rVyHx9vLuT3b21naGIYN81II9bRfyejm4wmUkITSQlN5Irk\nWTS2NXGg9hBmfyivqaGl/QhHnEdocR7hSHvnv1+8fcTZSlNbM1UtNbR1tJ3yuQwYsPmF4fC347CG\nd/579Ha4nw0fU/fXrXO5XDS1N1PXWk/dkXpqW+s6b7fWU3ekAX+zLxfFTiA2MLrbzyUiPatr/pk6\nOIqIeAQVaNLrfMwm5s0cyNSRMbz5cR47D1Tx2CsbuHhUDHMnJxPo790LW/eEAB8rGeHpOBxBlJfX\nn9W+zg4nR5ytXQVdc3sL1S3VlDdXUt5USXlzBeXNleypzmVPde5x+xowEOobgsO/82qbw2rH4R/e\n9b3JYPy80Gqtp/ZI/XHf17XW09DeQG1zHe0u5ylzri7MZHBYGtPjJ5NuG4jRoMVwRTxBV4t9XUET\nEfEIKtCkz8Q6AvnBDSPYtq+SxSty+XhzIet3lnL1RUlMGx2L2aQP7OfCZDRhNfpj9fnC4uAhCSc8\nrqX9CBXNlZ2FW3PFccVbTs0+cmr2nfVzmw0mQv1DiA2KIdgSRLAliBBLEMG+QV3fB1uCKGosYcXh\n1V1FYqQ1gmkDLuKCqNFYTJbu/Pgi0k1qsS8i4llUoEmfMhgMjEwNJyPJxopNBby79iD/WJHLJ1sL\nuXF6GsNT1H6+t/iZfYkLiiEuKOaEba3ONipbqihvqjhawFVS3lSBCxfBlmCCfQMJsQR/XnQdLcCs\nZn8iIoJPe9XP7m9jWPgQ8usLWZn/KRtLt/Lm3rd5b/9HTI6ZwJS4SYT4BvfWjy4ip1BQ0UiAn5nQ\nQP2xRETEE6hAE7cwm4xcOj6eiRlRvLN6P6u2FfHsv7YxLNnOTTNSibZrqE1fsph8iA6IJDogslef\nZ0BQLLcMuZGrUmazpiCTNYXr+ejQx/zv8CrGRo5k2oDJDDhJASkivaOt3UlZdRNpsSH9snmTiIgn\nUoEmbhVstXDL7MFMGx3Hmyty2bG/kl0Hq5g2OparL0oiwE/z085Hob4hXJkym0sTp5NVspmV+Z+S\nVbKJrJJNDAxNYXr8ZIbaB2uemkgvK65swuWiXzdtEhHxNCrQxCMMiAjkhzeNZEtuBYs/zmX5xgLW\n7yxl7uQkpo6MwWTUB/XzkcVkYXLsBC6MGc+uyr2szP+UPdW55NTsI8I/vHOeWvRYfDVPrc+4XC6K\nG0tpaGtgYFiqu+NILzs2/yxGHRxFRDyGCjTxGAaDgdEDHQxLtrN8Yz7vrTvIa8tyWLmlkJtmpDE0\n0ebuiNJLjAYjGeHpZISnU9hQzMf5a9hYsoXFOe/w3v6lXBQ7galxkwj1DXF31PPOsYIst2Y/udX7\nyK3ZT0Nb54f2xyYsIMIa7uaE0psKKjo7OMapg6OIiMdQgSYex8dsZM6EBCZlRPH26v18ur2Y3765\nlZGp4dw4PZVIm9XdEaUXxQZG8430G7g6ZQ6rCzJZU5jJskMrWX54FcPDhxIbGIXdz4bd30a4v41g\nS5DHD4XscHXQ1tFO+5e/XE7aOtpo73DiZ/Il3N+Gn9mvV7O4XC5KmsrIrd5HztGi7FhBBp3DT8dH\njWaofTAOfzXtOd/pCpqIiOdRgSYeKyTQl29dls700XH8Y3kOW/Mq2LG/kpnjBnDlpET8ffX2PZ8F\nW4K4InkWlyZMY0PpFlbkr2Fr+Q62lu847nFmoxmbX2hn0eYXht3fht2vs3iz+9kI8LGeU/MDZ4eT\npvZmGtoaaWhtoKGt6ejtRhrbGjtvtzXS6mylvcNJu+vzwqvtS0VYh6vjjJ830CcAh78du7/t6L/2\no+vS2bC7zv5DtMvlorSpjJyjBVle9X7q2xq6tof6hjAucjQDw5JJC00h3N+mZhH9SGF5IyEBFoKs\nGkYsIuIp9AlXPF5CVBAPfX00G/eW88+P8/go6zDrd5Zw4/Q0xqdH6MPkec7H5MOkmPFMjB5HeXMF\nFc1VVLZUUdlcfdy/ZU05J93f12Q5esUtrOvKW6hvCL4NRoqrKrsKrvq2LxRerY00tTefcUaz0YyP\n0YzZYMZsNONrshDgYz3h/s4v0+f3f2FbU3szFc2VVDRXcqi+gAN1h0/6Wth9wwj3txF+dDHxY7ft\nfjYsJp8vFGT7ya3ZR+5JC7JRpIUlMzA0VQVZP9Z8pJ3KuhaGJIa5O4qIiHyBCjTxCgaDgXGDIxiR\nYufD9Yf4cP1hAGJbYgAAIABJREFU/vzuTlZtLeTrMweqA1k/YDAYiLA6iLA6Trq9pb2FypZqKpur\nOv89roiroqix5LTPYTQYCfCxEuIbTGxgNAE+AQRaAgj06fwK8LES5BNIgMV69PsALEafHi9wnB1O\nao7UUtFc1Vm0tVRR3lxJbVsNJfXllDSVnXS/EEswHa6O4wqyEEtwV0GWFpqCw9+ugkwAKKrUAtUi\nIp5IBZp4FYuPibmTk5mUEcU/lueybV8ljy/awIwxcVx9UZKGPfZjfmY/YgOjiQ2MPmGby+Wisb2J\nquZqKlqqqD1SR3hoCK4WI4GWQAJ9OgsuP7OfR8xnMxlNnUM1/W0M4vNOig5HEOXl9TS1NVHR3Fm0\nVTZXUdFSSXlzFZXNlXS4jIyNHMnA0BTSwpJx+IerIJOTOjb/LFYNQkREPIo+zYpXigiz8r2vjWBr\nXgX/WJ7Dsg35ZO0u5YZpqUwY0ruLLYv3MRgMXVfB4oPjgM+LHW9k9bES72Pt+llEzkVXgaYGISIi\nHsX9fyoW6YaRqeH84o4LmHtREk0t7Sx8bxdPvrGFg8V17o4mIuLRCo+22FcHRxERz6ICTbyej9nE\nVRcl8Ys7LmBUWjg5+TV875lPeGN5Dk0t7e6OJyLikQrLG7EH+2louIiIh1GBJucNR6g/9103nO9/\nbQSRNivLNxbw44XrWbujGJfL5e54IiIeo6G5jdrGVs0/ExHxQCrQ5LwzPMXO8z+cxjVTkmk50s7L\nH+zmV69v5nCpd843EhHpaYXlncMbVaCJiHgeFWhyXrL4mLhyUiK/+PYFjBnoIK+glp/9dQOv/y+H\nppY2d8cTEXGrwgo1CBER8VQq0OS8Fh7izz3XDuMHN44gIszKik0FPPLSetZsL6JDwx5FpJ/6vIOj\n1kATEfE0KtCkX8hIsvPz28Zz3dRkjrQ5WfThHn756ib2Hq52dzQRkT5XWN6AwQDRdqu7o4iIyJeo\ndZP0Gz5mI5dPTGTi0CgWf5zHhj1lPPnGFjKSbFw3NYWEqCB3RxQR6XUul4vCikYiwqxYfEzujiMi\nIl+iAk36HVuwH9+dm8GlRXW8tWof2QeqyD5QxdjBEVwzOYlou+ZkiMj5q6ahlcaWdgbHh7k7ioiI\nnIQKNOm3kmOC+dHNo9h1sIq3Vu1j454yNu8t58JhUVx9URK2YD93RxQR6XFFxxqEqIOjiIhHUoEm\n/d6QRBvpCWFszqng7dX7WLO9mMydpUwfHcvlExMIslrcHVFEpMcca7Efow6OIiIeSQWaCGAwGBgz\nyMGotHAyd5bwzpr9LNuQz+ptRVw6Pp5Z4wbg76v/LiLi/Qq6rqCpg6OIiCfSJ06RLzAaDVw4LJrx\n6ZF8srWQ99cd5D+fHmDFpgKumJTItFEx+Jg1qV5EvFdheSMmo4HIMH93RxERkZNQm32Rk/AxG5k5\ndgC/vnMicycn4ezo4M0VuZ1rqG0rwtnR4e6IIiJnrcPloqiikWi7FbNJHwFERDyRfjuLnIK/r5mr\nLkziybsmMXt8PPVNbSz67x5++vJnbNxThkuLXYuIF6msbeFIm1PDG0VEPJiGOIqcgUB/H26YnsrM\ncQN4d+0B1mwr5o/vZJMQFcS1U5IZmmTDaDC4O6aIyCkVHpt/pgYhIiIeSwWayFkIC/Ll1tmDmT0+\nnn+v2c9nu8v43T+3ER7ix6SMKCZmRBEZZnV3TBGRkzrWwVEFmoiI51KBJnIOIm1W7ro6gzkX1PO/\njfls2lvOu2sP8u7ag6TGhjApI4px6REE+Pm4O6qISJdCrYEmIuLxVKCJdENCVBB3XDGEb8xysimn\njHXZJew+WE1eYS1vLM9hZGo4kzKiyUi2aUK+iLhdYXkjFrOR8FB1cBQR8VQq0ER6gK/FxKSMaCZl\nRFNV18L6XaWs3VHMxr3lbNxbTqC/DxOGRDJpWBQJkUEYNF9NRPqYs6OD4spG4hyBmjMrIuLBVKCJ\n9DBbsB+XTUhgzgXxHCqtZ92OEtbvKmX5pgKWbyogJjyASRlRTBgSiS3Yz91xRaQHNDc38/DDD1NZ\nWcmRI0e4++67GTx4MAsWLMDpdOJwOHjqqaewWCxuy1hW3Uy706XhjSIiHk4FmkgvMRgMJEYFkxgV\nzA3TU8neX8W67GK25lWw5JN9vPXJPtITw5iUEcXogQ78LPrvKOKtVq5cSUZGBt/+9rcpLCzktttu\nY/To0cybN485c+bwzDPPsGTJEubNm+e2jIXlxzo4qsW+iIgn0ydCkT5gNhkZmRbOyLRwGlva2LC7\nc77aroPV7DpYja9PDmMGObhySgoRQRYNgRTxMpdddlnX7eLiYiIjI8nKyuJnP/sZANOmTeOVV15x\nb4GmBiEiIl5BBZpIHwvw8+HiUbFcPCqW0qom1mWXkLmzhHXZnV/RditTRsQwKSOKIKv7hkOJyNm7\n6aabKCkp4cUXX+Rb3/pW15BGu91OeXm5W7Opxb6IiHdQgSbiRpE2K9dMSebqyUnsPVxD1p4y1m0v\nYvHHeby1ah9jBkUwdUQMg+JDdVVNxAu8+eab7N69mx/96Ee4XK6u+794+1TCwqyYzaZu53A4gk64\nr6S6mQA/MwOTwz3298nJcns6b8wM3pnbGzODd+b2xszgvbm/TAWaiAcwGgykJ4QxZWw8+w9Vsi67\nhFVbi8jaVUrWrlIibVamjohh0rAognVVTcTjZGdnY7fbiY6OJj09HafTSUBAAC0tLfj5+VFaWkpE\nRMRpj1Nd3dTtLA5HEOXl9cfd19bupKi8keTYYCoqGrr9HL3hZLk9nTdmBu/M7Y2ZwTtze2Nm8L7c\npyomz2hhppycHC655BJee+21E7atW7eO66+/nhtvvJEXXnjh3FOKCABBVguXjo/n/759AQ/NG8WE\noZFU1rbwz5V5PPj8Wl78Tza7D1bRcYZ/kReR3rdx40ZeeeUVACoqKmhqamLSpEksXboUgGXLljF5\n8mS35SuubKLD5SJOwxtFRDzeaa+gNTU18cQTTzBx4sSTbv/FL37Byy+/TGRkJPPnz+fSSy8lNTW1\nx4OK9DcGg4FB8WEMig9j3iVtZGaX8MnWQj7bXcZnu8uICPNn6ogYLhwWTXCArqqJuNNNN93E//t/\n/4958+bR0tLCT3/6UzIyMnjooYdYvHgxMTExzJ071235Pm8Qog6OIiKe7rQFmsViYeHChSxcuPCE\nbfn5+YSEhBAdHQ3A1KlTyczMVIEm0sMC/X2YOW4Al4yNI6+wllVbi9iwp4x/fbKPt1fvZ9RAB1NH\nxpCeEKYFaEXcwM/Pj9/+9rcn3L9o0SI3pDlR0dECLUZX0EREPN5pCzSz2YzZfPKHlZeXY7PZur63\n2Wzk5+f3XDoROY7BYCAtLpS0uFBuviSNzOwSVm0rYuOeMjbuKcMR6seUo1fVQgN93R1XRDxE1xpo\narEvIuLx+rxJSG92qPIGyt13vDEznHluB5A4wMZNs9PZe6iaj9YfZM3WIt5atZ9/r97PyIERTBs7\ngAkZUb2+CPb5/lp7Em/MLO5XUN5AsNVHTYZERLxAtz61RUREUFFR0fX9mXSp6q0OVd5AufuON2aG\nc89tD/Dh6zPSuObCRNbvKmVddgmb95axeW8ZvhYT4wZFMCkjioHxoT0+BLK/vdbu5K2Zxb1aWtup\nqG0hPSHM3VFEROQMdKtAi4uLo6GhgYKCAqKioli5ciVPP/10T2UTkbNk9fNh+ug4po+Oo7iykcyd\nJWRml/DpjmI+3VGMPdiXCUOjmJQRRbRdQ51E+oOiis4/jGqBahER73DaAi07O5snn3ySwsJCzGYz\nS5cuZfr06cTFxTFz5kwef/xxHnzwQQAuu+wykpKSej20iJxetD2Aa6ekMHdyMrn5NazNLmHjnjI+\nyDzEB5mHSIoOZlJGFBcMiSTQ38fdcUWklxSWd657pvlnIiLe4bQFWkZGBq+++upXbh83bhyLFy/u\n0VAi0nOMX2jX//WZA9mSW8667BJ2HqjiQHEdb67IZXiKnUkZUQxPCcfHfEbLI4qIl+hqsR+uFvsi\nIt6gz5uEiIj7+PqYmDAkiglDoqhpOELW0flqW3Ir2JJbQYCfmfHpkUzKiCI5JhiDWvaLeL1CtdgX\nEfEqKtBE+qnQQF8uHR/PpePjOVxaT+bOEtbvLGXllkJWbikkMsyf0YMcDIwLJTUuhAA/DYMU8UaF\n5Q3Ygn2x+umULyLiDfTbWkSIjwwiPjKI6y9OYffB6s4ukDnl/Hf9Yf7LYaBz/krnGmwhpMWFEB7i\n7+bUInI6Dc1t1DS0MizZ7u4oIiJyhlSgiUgXk9FIRrKdjGQ7La3t7CusI7eghpz8GvYX1VFY3sgn\nWwoBsAX7khYXyqjBkUSH+hEbHoDRqCGRIp6kqEILVIuIeBsVaCJyUn4WM0OTbAxNsgHQ7uzgcGkD\nuQU15BbUkltQQ9auUrJ2lQLg72smNTak6wpbUnQwFp/uL0ovIueuq4Oj5p+JiHgNFWgickbMJiPJ\nMcEkxwRz6XhwuVyUVjdTUtPC5t2l5BbUsGN/JTv2VwJgMhpIjA4iLS6UQQNCGZwQhq8KNpE+Vagr\naCIiXkcFmoicE4PBQJTNyrBBkYxM7rzKVtvYSl5BDTn5nVfYDhTVs6+wjo+yDmMxG0lPCGNEajgj\nUsMJC/J1808gcv4rLG/EAFqYXkTEi6hAE5EeExJgYcygCMYMigDonMdWVMeuA1Vs21fZ9cXSvcRH\nBjLyaLGWEBWEUS39RXqUy+WisKIRR5i/rl6LiHgRFWgi0mv8LGaGJtoYmmjja9NSKatpZlteBdvz\nKthzuIbDpQ28u/YgIQEWRqTaGZESzpBEG74WfZgU6a66xlYamttIiwtxdxQRETkLKtBEpM9EhPoz\nc+wAZo4dQPORdnYeqGJbXgXb9lWyelsxq7cVYzZ1DoUcmWpneEo49hA/d8cW8UoFXfPPAt2cRETE\nO3zyyQouvnjGaR/33HO/5Wtfu4mYmNheyaECTUTcwt/XzNjBEYwdHEFHh4v9xXWdxVpexReajeQw\nICKw8+paajhJ0cEaCilyhgrLOwu0ODUIERE5reLiIpYvX3pGBdr3vvdgr2ZRgSYibmc0GkiNDSE1\nNoTrpqZQUdvMtrxKtu2rYM+havLLGnh/3SGCrD4MSbQxJCGMoUk2bMG6uibyVY612I9Ri30RkdN6\n5pkn2b17J5Mnj2PWrDkUFxfx7LN/5Fe/+jnl5WU0Nzdz223f4cILJ3Pvvd/hBz9YwMqVK2hsbODw\n4UMUFhZw//0PMnHihd3OogJNRDxOeIg/M8bEMWNMHC2t7ew6WN05d21/5XFrr0XarAxJDGNooo3B\n8aFY/XzcnFzEcxRVNGIydnZbFRHxJv/8OI8Ne8rOah+TyYDT6frK7eMGR3DD9NSv3H7zzd/g7bf/\nSVJSCocPH+SPf/wL1dVVjB8/gTlzrqCwsIBHH32YCy+cfNx+ZWWlPP3071m/fh3/+c9bKtBE5Pzn\nZzEzeqCD0QMduFwuiiqb2HWwil0HqtiTX8PKzYWs3FyIwQBJ0cEMSbQxNDGMlNgQzCaju+OLuIXL\n5aKgopEom1X/D0REzlJ6+lAAgoKC2b17J++++zYGg5G6utoTHjt8+EgAIiIiaGho6JHnV4EmIl7D\nYDAQGx5AbHgAM8cOoN3ZwYHiOnYeqGLXoWr2F9axv6iO99cdxOJjZNCAsK4rbLGOAAyavyb9RGVd\nC0danVqgWkS80g3TU095tetkHI4gysvre+T5fXw6R+T8738fUVdXxwsv/IW6ujruuOMbJzzWZPq8\n87TL9dVX8M6GCjQR8Vpmk5G0uFDS4kKZOxmaj7Sz93ANuw5WsfNg1ReajUBwgIUhCWGdc9gSwwgP\nV2c7OX8daxASq/lnIiJnxGg04nQ6j7uvpqaG6OgYjEYjq1Z9TFtbW59kUYEmIucNf18zI9PCGZkW\nDkB1/ZHO4ZAHq9h1sJr1u0pZf3T+WoC/D5Fh/kTZrJ9/2a1EhvnjY9Y6bOLdCtViX0TkrCQkJLF3\n7x6io2MIDQ0F4OKLp/Pwwz9g165sLr/8KiIiIli0aGGvZ1GBJiLnrbAgXy4cFs2Fw6I7569VNLLz\nYDV7D1dTXtvCoZJ69hfVHbePAbCH+B1XtB27HRbkq2GS4hWOdXDUFTQRkTMTFhbG229/cNx90dEx\n/O1vb3Z9P2vWHAC+9a1vA5Cc/PkwzOTkVJ5//qUeyaICTUT6BYPBQKwjkFhHILPGDcDhCKKktJaK\nmhaKq5ooqWyipOrzr+wDVWQfqDruGL4+JiJtx191S4sL1WLa4nEKKxrxMRtxhPq7O4qIiJwlFWgi\n0m+ZjEYibVYibVb40lzkppZ2Sqs7C7fiY4VbZRPFlU0cLj2+S1OcI5CRaXZGpISTFKPFtMW9nB0u\niiqaiA0PwGjUe1FExNuoQBMROQmrn5mk6GCSooOPu7/D5aKqroWSqiaKyhvJPljFnkPVvL+uczHt\nYKsPw1PCGZFqZ2iSDT+Lfs1K3yqpbKTd2aEOjiIiXkqfHEREzoLRYCA8xJ/wEH8ykuzMGh/ftZj2\n1rwKtu+r5NMdxXy6oxizycDg+DBGpHYWbOEhGm4mve9Qcee8ShVoIiLeSQWaiEg3fXEx7Q6Xi4PF\n9WzLq2BbXkXXXLbX/9f5gXlkajgjUsNJjg7W8DPpFYdKOtcBitVSEiIiXkkFmohIDzIaDCTHBJMc\nE8w1U5Kpqmth275KtuVVsOtgNR9kHuKDzEMEWX0YnmxnRGo4Q5Ns+Pvq17H0jEMlR6+gqYOjiIhX\n0icCEZFeZAv2Y9qoWKaNiuVIq5Ndh6rYllfJtn0VrM0uYW12CSajgbS4EAYOCGXQgFCSY0Pw9dFa\nbHJuDpfU42cxYQv2dXcUEZHzzvXXX8nf/74Yq9Xaa8+hAk1EpI/4WkyMSnMwKq1zKOTh0nq25law\nbV8lew/XsOdwDQAmo4HE6KCjBVsYqbEhWP3061pOr629g6LyBhKjg7Rmn4iIl9IZX0TEDYwGA4lR\nwSRGBTN3cjKNLW3kFtSSc7iGvfk1HCiqZ19hHf9dfxiDAeIjghgxyEF8eABpcSEEWS3u/hHEA5VW\nNeHscGn+mYjIWbrttq/zy1/+lqioKEpKinnkkQdxOCJobm6mpaWFBx74EUOGZPRJFhVoIiIeIMDP\nh5Gp4YxMDQegpbWdfYV17M2vJudwDfuL6zhUWt/1+NjwAAbGdw6JHDgglNBADWcTKKjoXKNPHRxF\nxJu9nfc+W8p2nNU+JqMBZ4frK7ePihjGtalXfOX2KVOmsXbtaq677gbWrFnFlCnTSElJY8qUi9m0\naQOvv/43/u//njqrTOdKBZqIiAfys5gZmmRjaJINgLZ2J1VN7WTtKCInv4a8wloKNzeycnMhABFh\n/gwaEEp8ZBBWPzP+FjP+vib8jv3r23mfj9nozh9LellheSMAcWoQIiJyVqZMmcbzzz/LddfdwKef\nruLeex/gzTdf5R//eJW2tjb8/Pz6LIsKNBERL+BjNpGREkrk0cYP7c4ODpXUk5PfOSQyt6CGNduL\ngeJTHsdsMnQVbf4W89HCzYS/7+e3/XzNOEL8GD8kEqPmMXmVYwVajENDHEXEe12besUpr3adjMMR\nRHl5/ekf+BWSk1OorCyntLSE+vp61qz5hPDwCB599An27NnF888/e87HPlsq0EREvJDZZCQlNoSU\n2BDmTEigo8NFflkDxVWNtLQ6aTnipPlIO82t7Z23W9tpOdJOc2vn/S2tTspqmmlpdX7lc6TEhuAI\n1eLa3qSxpY3wUH+CrT7ujiIi4nUmTryIl176I5MnT6WmppqUlDQAVq1aSXt7e5/lUIEmInIeMBoN\nJEQFkRAVdFb7dbhcHGk9Vsw5jxZx7VjMJhVnXujOq4YSEmrF0NHh7igiIl5n6tRp3HXXbfz1r/+g\npaWZX/ziMVauXM51193A8uXL+OCDd/skhwo0EZF+zGgw4O9r1kLZ5wlbsB8Oe0C3hvmIiPRX6elD\nWbUqq+v7119f0nX7ooumAnD55Vf1eg6dkUVERHrAb37zGzZt2kR7ezt33nknw4YNY8GCBTidThwO\nB0899RQWi5ZHEBGRU1OBJiIi0k3r168nNzeXxYsXU11dzTXXXMPEiROZN28ec+bM4ZlnnmHJkiXM\nmzfP3VFFRMTDqd+yiIhIN40bN47nnnsOgODgYJqbm8nKymLGjBkATJs2jczMTHdGFBERL6EraCIi\nIt1kMpmwWq0ALFmyhClTpvDpp592DWm02+2Ul5ef9jhhYVbMZlO38zgcZ9csxlN4Y25vzAzemdsb\nM4N35vbGzOC9ub9MBZqIiEgPWb58OUuWLOGVV15h1qxZXfe7XK4z2r+6uqnbGbq7FpC7eGNub8wM\n3pnbGzODd+b2xszgfblPVUxqiKOIiEgPWLNmDS+++CILFy4kKCgIq9VKS0sLAKWlpURERLg5oYiI\neAMVaCIiIt1UX1/Pb37zG/785z8TGhoKwKRJk1i6dCkAy5YtY/Lkye6MKCIiXkJDHEVERLrpww8/\npLq6mu9///td9/3617/mJz/5CYsXLyYmJoa5c+e6MaGIiHgLFWgiIiLddOONN3LjjTeecP+iRYvc\nkEZERLyZhjiKiIiIiIh4CIPrTFtLiYiIiIiISK/SFTQREREREREPoQJNRERERETEQ6hAExERERER\n8RAq0ERERERERDyECjQREREREREPoQJNRERERETEQ3j8QtW//OUv2bZtGwaDgR//+McMHz68a9u6\ndet45plnMJlMTJkyhXvuuceNSY/3m9/8hk2bNtHe3s6dd97JrFmzurZNnz6dqKgoTCYTAE8//TSR\nkZHuigpAVlYW3/ve90hLSwNg4MCBPProo13bPfW1/te//sW7777b9X12djZbtmzp+n7o0KGMHj26\n6/u//vWvXa97X8vJyeHuu+/mm9/8JvPnz6e4uJgFCxbgdDpxOBw89dRTWCyW4/Y51fvfnbkfeeQR\n2tvbMZvNPPXUUzgcjq7Hn+695I7MDz/8MDt37iQ0NBSA22+/nYsvvvi4fTzxtb7//vuprq4GoKam\nhpEjR/LEE090Pf7tt9/mueeeIz4+HoBJkybx3e9+t89zi3vo/Ng3dH7sGzpHui+zzpEeyOXBsrKy\nXN/5zndcLpfLlZeX57rhhhuO2z5nzhxXUVGRy+l0um6++WZXbm6uO2KeIDMz03XHHXe4XC6Xq6qq\nyjV16tTjtk+bNs3V0NDghmRfbf369a777rvvK7d76mv9RVlZWa7HH3/8uPvGjx/vpjTHa2xsdM2f\nP9/1k5/8xPXqq6+6XC6X6+GHH3Z9+OGHLpfL5frtb3/rev3114/b53Tv/75wstwLFixwffDBBy6X\ny+V67bXXXE8++eRx+5zuvdTbTpb5oYcecn388cdfuY+nvtZf9PDDD7u2bdt23H1vvfWW69e//nVf\nRRQPovNj39H5sffpHNl3dI70Dh49xDEzM5NLLrkEgJSUFGpra2loaAAgPz+fkJAQoqOjMRqNTJ06\nlczMTHfG7TJu3Diee+45AIKDg2lubsbpdLo51bnz5Nf6i1544QXuvvtud8c4KYvFwsKFC4mIiOi6\nLysrixkzZgAwbdq0E17TU73/+8rJcj/22GNceumlAISFhVFTU9OnmU7nZJlPx1Nf62P2799PfX29\nW/5iKZ5J50fP4Mmv9Rd58vkRdI7sSzpHegePLtAqKioICwvr+t5ms1FeXg5AeXk5NpvtpNvczWQy\nYbVaAViyZAlTpkw5YdjAY489xs0338zTTz+Ny+VyR8wT5OXlcdddd3HzzTezdu3arvs9+bU+Zvv2\n7URHRx83jACgtbWVBx98kJtuuolFixa5KR2YzWb8/PyOu6+5ublruIbdbj/hNT3V+7+vnCy31WrF\nZDLhdDp54403uPLKK0/Y76veS33hZJkBXnvtNW655RYeeOABqqqqjtvmqa/1MX//+9+ZP3/+Sbd9\n9tln3H777dx6663s2rWrNyOKB9H5sW/p/Ni7dI7sOzpHegePn4P2RZ7yi/pMLV++nCVLlvDKK68c\nd//999/P5MmTCQkJ4Z577mHp0qXMnj3bTSk7JSYmcu+99zJnzhzy8/O55ZZbWLZs2QnjvT3VkiVL\nuOaaa064f8GCBVx11VUYDAbmz5/P2LFjGTZsmBsSntqZvLc96f3vdDpZsGABEyZMYOLEicdt88T3\n0tVXX01oaCjp6em89NJLPP/88/z0pz/9ysd70mvd2trKpk2bePzxx0/YNmLECGw2GxdffDFbtmzh\noYce4r333uv7kOJ2nvSePRM6P/Ydbz8/gs6RvU3nSM/j0VfQIiIiqKio6Pq+rKys6y9AX95WWlp6\nVpdre9uaNWt48cUXWbhwIUFBQcdtmzt3Lna7HbPZzJQpU8jJyXFTys9FRkZy2WWXYTAYiI+PJzw8\nnNLSUsDzX2voHAoxatSoE+6/+eabCQgIwGq1MmHCBI94rY+xWq20tLQAJ39NT/X+d7dHHnmEhIQE\n7r333hO2neq95C4TJ07k/7dz7y6NRFEcx38RQ3w2CgpiIyl8dEFEMIggxMK/wS6VoIKoYCPaDeoU\nwhRi1CKlnQQbbWyFiI2P0kYFm7HQNOpAthgcdtUsW829i99PmTvF4XK4Z05yc/r7+yWFQwg+54HN\ne10ul2te20in09EfuTOZjJ6env7r62L4d9TH+FAfzaBGxocaaR+rG7RsNqvj42NJ0vX1tTo6OtTS\n0iJJ6u7uVqVS0f39vYIg0OnpqbLZrMlwIy8vL9rY2NDOzk40Eef3tXw+r7e3N0lhYn1M8jGpVCpp\nf39fUnhlw/f9aHKWzXsthQd3c3Pzl2+fbm9vtbCwoGq1qiAIdHFxYcVefxgZGYny++TkRKOjo3+s\n/y3/TSqVSkomk5qbm6u5XiuXTJmdndXd3Z2k8GXlcx7YuteSdHl5qb6+vm/Xdnd3dXR0JCmcbtXW\n1mZ0ChtUyZQnAAABhklEQVTiQ32MD/XRDGpkfKiR9klUbfqd8huu6+r8/FyJREKrq6u6ublRa2ur\ncrmcyuWyXNeVJE1MTCifzxuONnRwcCDP89TT0xN9Njw8rN7eXuVyORWLRR0eHiqVSmlgYEArKytK\nJBIGI5YqlYoWFxf1/Pys9/d3zczMyPd96/daCkcHb21taW9vT5JUKBQ0NDSkTCajzc1NnZ2dqa6u\nTuPj48bGq15dXWl9fV0PDw+qr69XZ2enXNfV8vKyXl9f1dXVJcdxlEwmNT8/L8dx1NDQ8CX/ax1C\nccbt+75SqVR0OKfTaa2trUVxB0HwJZfGxsaMxjw1NaVCoaDGxkY1NTXJcRy1t7dbv9ee58nzPA0O\nDmpycjJ6dnp6Wtvb23p8fNTS0lL0kmVq9DHMoD7Gg/oYT5zUSHMxUyPtY32DBgAAAAA/hdVXHAEA\nAADgJ6FBAwAAAABL0KABAAAAgCVo0AAAAADAEjRoAAAAAGAJGjQAAAAAsAQNGgAAAABYggYNAAAA\nACzxC/7gtb1qSzPnAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f7071c1cef0>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "BWGzMSaBnYMb",
        "colab_type": "code",
        "outputId": "a6efc12c-b694-4741-ef6c-1b3c6caadb02",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "# Test performance\n",
        "trainer.run_test_loop()\n",
        "print(\"Test loss: {0:.2f}\".format(trainer.train_state['test_loss']))\n",
        "print(\"Test Accuracy: {0:.1f}%\".format(trainer.train_state['test_acc']))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test loss: 1.28\n",
            "Test Accuracy: 67.2%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "5672VEginYnY",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Save all results\n",
        "trainer.save_train_state()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "HN1g2vP3nad_",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Inference"
      ]
    },
    {
      "metadata": {
        "id": "Myr8QQjKnZ7k",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "class Inference(object):\n",
        "    def __init__(self, model, vectorizer):\n",
        "        self.model = model\n",
        "        self.vectorizer = vectorizer\n",
        "  \n",
        "    def predict_nationality(self, surname):\n",
        "        # Forward pass\n",
        "        vectorized_surname = torch.tensor(self.vectorizer.vectorize(surname)).unsqueeze(0)\n",
        "        self.model.eval()\n",
        "        y_pred = self.model(vectorized_surname, apply_softmax=True)\n",
        "\n",
        "        # Top nationality\n",
        "        y_prob, indices = y_pred.max(dim=1)\n",
        "        index = indices.item()\n",
        "\n",
        "        # Predicted nationality\n",
        "        nationality = vectorizer.nationality_vocab.lookup_index(index)\n",
        "        probability = y_prob.item()\n",
        "        return {'nationality': nationality, 'probability': probability}\n",
        "  \n",
        "    def predict_top_k(self, surname, k):\n",
        "        # Forward pass\n",
        "        vectorized_surname = torch.tensor(self.vectorizer.vectorize(surname)).unsqueeze(0)\n",
        "        self.model.eval()\n",
        "        y_pred = self.model(vectorized_surname, apply_softmax=True)\n",
        "\n",
        "        # Top k nationalities\n",
        "        y_prob, indices = torch.topk(y_pred, k=k)\n",
        "        probabilities = y_prob.detach().numpy()[0]\n",
        "        indices = indices.detach().numpy()[0]\n",
        "\n",
        "        # Results\n",
        "        results = []\n",
        "        for probability, index in zip(probabilities, indices):\n",
        "            nationality = self.vectorizer.nationality_vocab.lookup_index(index)\n",
        "            results.append({'nationality': nationality, 'probability': probability})\n",
        "\n",
        "        return results"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "vV2SBrXpdllN",
        "colab_type": "code",
        "outputId": "b46c411d-7de6-4afa-a3bd-5a073a67550c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 187
        }
      },
      "cell_type": "code",
      "source": [
        "# Load the model\n",
        "dataset = SurnameDataset.load_dataset_and_load_vectorizer(\n",
        "    args.split_data_file, args.vectorizer_file)\n",
        "vectorizer = dataset.vectorizer\n",
        "model = SurnameModel(num_input_channels=len(vectorizer.surname_vocab),\n",
        "                     num_output_channels=args.num_filters,\n",
        "                     num_classes=len(vectorizer.nationality_vocab),\n",
        "                     dropout_p=args.dropout_p)\n",
        "model.load_state_dict(torch.load(args.model_state_file))\n",
        "model = model.to(\"cpu\")\n",
        "print (model.named_modules)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Reloading!\n",
            "<bound method Module.named_modules of SurnameModel(\n",
            "  (conv): ModuleList(\n",
            "    (0): Conv1d(28, 100, kernel_size=(2,), stride=(1,))\n",
            "    (1): Conv1d(28, 100, kernel_size=(3,), stride=(1,))\n",
            "    (2): Conv1d(28, 100, kernel_size=(4,), stride=(1,))\n",
            "  )\n",
            "  (dropout): Dropout(p=0.1)\n",
            "  (fc1): Linear(in_features=300, out_features=18, bias=True)\n",
            ")>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "TRc5KCZinaBh",
        "colab_type": "code",
        "outputId": "cdcbf2ad-4576-4a88-9c23-32bae30bc4de",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "# Inference\n",
        "inference = Inference(model=model, vectorizer=vectorizer)\n",
        "surname = input(\"Enter a surname to classify: \")\n",
        "prediction = inference.predict_nationality(preprocess_text(surname))\n",
        "print(\"{} → {} (p={:0.2f})\".format(surname, prediction['nationality'], \n",
        "                                    prediction['probability']))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Enter a surname to classify: Goku\n",
            "Goku → Japanese (p=0.79)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "P5slsQKwnZ_H",
        "colab_type": "code",
        "outputId": "a4dfae91-cb3a-47e3-8d98-71680f12fa5a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 323
        }
      },
      "cell_type": "code",
      "source": [
        "# Top-k inference\n",
        "top_k = inference.predict_top_k(preprocess_text(surname), \n",
        "                                k=len(vectorizer.nationality_vocab))\n",
        "for result in top_k:\n",
        "    print (\"{} → {} (p={:0.2f})\".format(surname, result['nationality'], \n",
        "                                         result['probability']))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Goku → Japanese (p=0.79)\n",
            "Goku → Korean (p=0.08)\n",
            "Goku → Vietnamese (p=0.04)\n",
            "Goku → Czech (p=0.03)\n",
            "Goku → Polish (p=0.02)\n",
            "Goku → Russian (p=0.01)\n",
            "Goku → Arabic (p=0.01)\n",
            "Goku → Greek (p=0.00)\n",
            "Goku → German (p=0.00)\n",
            "Goku → Chinese (p=0.00)\n",
            "Goku → English (p=0.00)\n",
            "Goku → Dutch (p=0.00)\n",
            "Goku → Scottish (p=0.00)\n",
            "Goku → Irish (p=0.00)\n",
            "Goku → Spanish (p=0.00)\n",
            "Goku → Portuguese (p=0.00)\n",
            "Goku → French (p=0.00)\n",
            "Goku → Italian (p=0.00)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "HQSsKNRSxjRB",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# Batch normalization"
      ]
    },
    {
      "metadata": {
        "id": "r3EamVazx2hx",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "Even though we standardized our inputs to have zero mean and unit variance to aid with convergence, our inputs change during training as they go through the different layers and nonlinearities. This is known as internal covariate shirt and it slows down training and requires us to use smaller learning rates. The solution is [batch normalization](https://arxiv.org/abs/1502.03167) (batchnorm) which makes normalization a part of the model's architecture. This allows us to use much higher learning rates and get better performance, faster.\n",
        "\n",
        "$ BN = \\frac{a - \\mu_{x}}{\\sqrt{\\sigma^2_{x} + \\epsilon}}  * \\gamma + \\beta $\n",
        "\n",
        "where:\n",
        "* $a$ = activation | $\\in \\mathbb{R}^{NXH}$ ($N$ is the number of samples, $H$ is the hidden dim)\n",
        "* $ \\mu_{x}$ = mean of each hidden | $\\in \\mathbb{R}^{1XH}$\n",
        "* $\\sigma^2_{x}$ = variance of each hidden | $\\in \\mathbb{R}^{1XH}$\n",
        "* $epsilon$ = noise\n",
        "* $\\gamma$ = scale parameter (learned parameter)\n",
        "* $\\beta$ = shift parameter (learned parameter)\n",
        "\n"
      ]
    },
    {
      "metadata": {
        "id": "9koMITOdzfZB",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "But what does it mean for our activations to have zero mean and unit variance before the nonlinearity operation. It doesn't mean that the entire activation matrix has this property but instead batchnorm is applied on the hidden (num_output_channels in our case) dimension. So each hidden's mean and variance is calculated using all samples across the batch. Also, batchnorm uses the calcualted mean and variance of the activations in the batch during training. However, during test, the sample size could be skewed so the model uses the saved population mean and variance from training. PyTorch's [BatchNorm](https://pytorch.org/docs/stable/nn.html#torch.nn.BatchNorm1d) class takes care of all of this for us automatically.\n",
        "\n",
        "<img src=\"https://raw.githubusercontent.com/GokuMohandas/practicalAI/master/images/batchnorm.png\" width=400>"
      ]
    },
    {
      "metadata": {
        "id": "RsWdAKVEHvyV",
        "colab_type": "code",
        "colab": {}
      },
      "cell_type": "code",
      "source": [
        "# Model with batch normalization\n",
        "class SurnameModel_BN(nn.Module):\n",
        "    def __init__(self, num_input_channels, num_output_channels, num_classes, dropout_p):\n",
        "        super(SurnameModel_BN, self).__init__()\n",
        "        \n",
        "        # Conv weights\n",
        "        self.conv = nn.ModuleList([nn.Conv1d(num_input_channels, num_output_channels, \n",
        "                                             kernel_size=f) for f in [2,3,4]])\n",
        "        self.conv_bn = nn.ModuleList([nn.BatchNorm1d(num_output_channels) # define batchnorms\n",
        "                                      for i in range(3)])\n",
        "        self.dropout = nn.Dropout(dropout_p)\n",
        "       \n",
        "        # FC weights\n",
        "        self.fc1 = nn.Linear(num_output_channels*3, num_classes)\n",
        "\n",
        "    def forward(self, x, channel_first=False, apply_softmax=False):\n",
        "        \n",
        "        # Rearrange input so num_input_channels is in dim 1 (N, C, L)\n",
        "        if not channel_first:\n",
        "            x = x.transpose(1, 2)\n",
        "            \n",
        "        # Conv outputs\n",
        "        z = [F.relu(conv_bn(conv(x))) for conv, conv_bn in zip(self.conv, self.conv_bn)]\n",
        "        z = [F.max_pool1d(zz, zz.size(2)).squeeze(2) for zz in z]\n",
        "        \n",
        "        # Concat conv outputs\n",
        "        z = torch.cat(z, 1)\n",
        "        z = self.dropout(z)\n",
        "\n",
        "        # FC layer\n",
        "        y_pred = self.fc1(z)\n",
        "        \n",
        "        if apply_softmax:\n",
        "            y_pred = F.softmax(y_pred, dim=1)\n",
        "        return y_pred"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "metadata": {
        "id": "s_QcGx4vN3bQ",
        "colab_type": "code",
        "outputId": "cf7b1ffa-6520-43ab-f950-b26defc1bcea",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        }
      },
      "cell_type": "code",
      "source": [
        "# Initialization\n",
        "dataset = SurnameDataset.load_dataset_and_make_vectorizer(args.split_data_file)\n",
        "dataset.save_vectorizer(args.vectorizer_file)\n",
        "vectorizer = dataset.vectorizer\n",
        "model = SurnameModel_BN(num_input_channels=len(vectorizer.surname_vocab),\n",
        "                        num_output_channels=args.num_filters,\n",
        "                        num_classes=len(vectorizer.nationality_vocab),\n",
        "                        dropout_p=args.dropout_p)\n",
        "print (model.named_modules)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<bound method Module.named_modules of SurnameModel_BN(\n",
            "  (conv): ModuleList(\n",
            "    (0): Conv1d(28, 100, kernel_size=(2,), stride=(1,))\n",
            "    (1): Conv1d(28, 100, kernel_size=(3,), stride=(1,))\n",
            "    (2): Conv1d(28, 100, kernel_size=(4,), stride=(1,))\n",
            "  )\n",
            "  (conv_bn): ModuleList(\n",
            "    (0): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "    (1): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "    (2): BatchNorm1d(100, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
            "  )\n",
            "  (dropout): Dropout(p=0.1)\n",
            "  (fc1): Linear(in_features=300, out_features=18, bias=True)\n",
            ")>\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "tBXzxtiaxmXi",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "You can train this model with batch normalization and you'll notice that the validation results improve by ~2-5%."
      ]
    },
    {
      "metadata": {
        "id": "ERMGiPgAPssx",
        "colab_type": "code",
        "outputId": "950c476f-01a3-48b8-9c4c-da4b78eb3f2a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        }
      },
      "cell_type": "code",
      "source": [
        "# Train\n",
        "trainer = Trainer(dataset=dataset, model=model, \n",
        "                  model_state_file=args.model_state_file, \n",
        "                  save_dir=args.save_dir, device=args.device,\n",
        "                  shuffle=args.shuffle, num_epochs=args.num_epochs, \n",
        "                  batch_size=args.batch_size, learning_rate=args.learning_rate, \n",
        "                  early_stopping_criteria=args.early_stopping_criteria)\n",
        "trainer.run_train_loop()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[EPOCH]: 00 | [LR]: 0.001 | [TRAIN LOSS]: 2.66 | [TRAIN ACC]: 22.5% | [VAL LOSS]: 2.33 | [VAL ACC]: 38.6%\n",
            "[EPOCH]: 01 | [LR]: 0.001 | [TRAIN LOSS]: 2.05 | [TRAIN ACC]: 43.4% | [VAL LOSS]: 1.94 | [VAL ACC]: 44.3%\n",
            "[EPOCH]: 02 | [LR]: 0.001 | [TRAIN LOSS]: 1.64 | [TRAIN ACC]: 51.4% | [VAL LOSS]: 1.71 | [VAL ACC]: 59.4%\n",
            "[EPOCH]: 03 | [LR]: 0.001 | [TRAIN LOSS]: 1.39 | [TRAIN ACC]: 57.1% | [VAL LOSS]: 1.52 | [VAL ACC]: 61.7%\n",
            "[EPOCH]: 04 | [LR]: 0.001 | [TRAIN LOSS]: 1.21 | [TRAIN ACC]: 60.6% | [VAL LOSS]: 1.46 | [VAL ACC]: 57.3%\n",
            "[EPOCH]: 05 | [LR]: 0.001 | [TRAIN LOSS]: 1.09 | [TRAIN ACC]: 63.1% | [VAL LOSS]: 1.38 | [VAL ACC]: 59.2%\n",
            "[EPOCH]: 06 | [LR]: 0.001 | [TRAIN LOSS]: 0.98 | [TRAIN ACC]: 64.5% | [VAL LOSS]: 1.37 | [VAL ACC]: 67.4%\n",
            "[EPOCH]: 07 | [LR]: 0.001 | [TRAIN LOSS]: 0.90 | [TRAIN ACC]: 67.6% | [VAL LOSS]: 1.27 | [VAL ACC]: 63.9%\n",
            "[EPOCH]: 08 | [LR]: 0.001 | [TRAIN LOSS]: 0.81 | [TRAIN ACC]: 68.9% | [VAL LOSS]: 1.31 | [VAL ACC]: 63.6%\n",
            "[EPOCH]: 09 | [LR]: 0.001 | [TRAIN LOSS]: 0.77 | [TRAIN ACC]: 70.2% | [VAL LOSS]: 1.28 | [VAL ACC]: 63.8%\n",
            "[EPOCH]: 10 | [LR]: 0.001 | [TRAIN LOSS]: 0.67 | [TRAIN ACC]: 72.7% | [VAL LOSS]: 1.29 | [VAL ACC]: 70.1%\n",
            "[EPOCH]: 11 | [LR]: 0.001 | [TRAIN LOSS]: 0.65 | [TRAIN ACC]: 73.0% | [VAL LOSS]: 1.26 | [VAL ACC]: 68.6%\n",
            "[EPOCH]: 12 | [LR]: 0.001 | [TRAIN LOSS]: 0.63 | [TRAIN ACC]: 74.2% | [VAL LOSS]: 1.23 | [VAL ACC]: 70.3%\n",
            "[EPOCH]: 13 | [LR]: 0.001 | [TRAIN LOSS]: 0.61 | [TRAIN ACC]: 74.6% | [VAL LOSS]: 1.26 | [VAL ACC]: 69.1%\n",
            "[EPOCH]: 14 | [LR]: 0.001 | [TRAIN LOSS]: 0.59 | [TRAIN ACC]: 75.1% | [VAL LOSS]: 1.25 | [VAL ACC]: 71.6%\n",
            "[EPOCH]: 15 | [LR]: 0.001 | [TRAIN LOSS]: 0.54 | [TRAIN ACC]: 76.4% | [VAL LOSS]: 1.27 | [VAL ACC]: 70.2%\n",
            "[EPOCH]: 16 | [LR]: 0.001 | [TRAIN LOSS]: 0.53 | [TRAIN ACC]: 77.3% | [VAL LOSS]: 1.25 | [VAL ACC]: 70.2%\n",
            "[EPOCH]: 17 | [LR]: 0.001 | [TRAIN LOSS]: 0.51 | [TRAIN ACC]: 77.1% | [VAL LOSS]: 1.26 | [VAL ACC]: 69.6%\n",
            "[EPOCH]: 18 | [LR]: 0.001 | [TRAIN LOSS]: 0.51 | [TRAIN ACC]: 77.1% | [VAL LOSS]: 1.26 | [VAL ACC]: 72.2%\n",
            "[EPOCH]: 19 | [LR]: 0.001 | [TRAIN LOSS]: 0.50 | [TRAIN ACC]: 77.5% | [VAL LOSS]: 1.27 | [VAL ACC]: 72.1%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "iiAW6AL0QAJ8",
        "colab_type": "code",
        "outputId": "8409aecf-1c40-4a86-9bb9-38a6bf31affd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 335
        }
      },
      "cell_type": "code",
      "source": [
        "# Plot performance\n",
        "trainer.plot_performance()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2gAAAE+CAYAAAD4XjP+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd8VOed/v3P9FHXSBr1AuoCJHoH\nYcB0bLCxDe5rP2m/deJ415v1xps4duquEyebxI6duDuOS4ypBptmigyIIkA0NRAIdY2kGdWRpj5/\nCGQwHUmMRnzfrxeWdM7MOZeEJXTNuc99K9xutxshhBBCCCGEEB6n9HQAIYQQQgghhBBdpKAJIYQQ\nQgghRD8hBU0IIYQQQggh+gkpaEIIIYQQQgjRT0hBE0IIIYQQQoh+QgqaEEIIIYQQQvQTUtCEuEFp\naWnU1NR4OoYQQghxUyxbtow777zT0zGEGPCkoAkhhBBCiCsqLi4mICCA6OhoDh486Ok4QgxoUtCE\n6GWdnZ0899xzzJkzh3nz5vE///M/OJ1OAN5//33mzZvH3LlzueeeeygpKbnidiGEEKI/WLlyJXPn\nzmXhwoWsWrWqe/uqVauYM2cOc+bM4Uc/+hE2m+2y2/fs2cOsWbO6n3v+x3/+85/5yU9+wj333MM7\n77yDy+XihRdeYM6cOcyYMYMf/ehH2O12ABobG/ne977HzJkzueOOO/jqq6/Ytm0bCxcuvCDz3Xff\nzebNm/v6SyNEr1N7OoAQA827775LTU0N69atw+Fw8NBDD/HZZ58xc+ZM/vjHP7J161b8/f35/PPP\n2bZtG1FRUZfcnpKS4ulPRQghhMDpdLJp0yaeeOIJVCoVL730Ejabjbq6Ov73f/+XVatWER4ezg9+\n8APee+895s6de8ntmZmZVzzP9u3bWb16NSEhIWzYsIH9+/fz2Wef4XK5uOuuu1i/fj2LFi3ipZde\nIikpiddee43jx4/z2GOPkZOTg8lkorCwkPT0dKqqqjhz5gzZ2dk36askRO+RgiZEL9u2bRuPP/44\narUatVrNHXfcwc6dO5k/fz4KhYLly5ezcOFC5s2bB4Ddbr/kdiGEEKI/+Oqrr8jMzMTf3x+AcePG\nsXXrViwWCyNHjiQiIgKAl156CZVKxaeffnrJ7Xl5eVc8z/DhwwkJCQFgzpw5TJ8+HY1GA0BmZibl\n5eVAV5F7/fXXARgyZAhbtmxBq9UyZ84c1q1bR3p6Ops3b2bmzJlotdre/4II0cdkiKMQvayxsZGg\noKDuj4OCgmhoaECj0fDOO+9w4MAB5syZwwMPPEBRUdFltwshhBD9wYoVK9i2bRtjxoxhzJgxbNy4\nkZUrV2I2mwkMDOx+nE6nQ61WX3b71Zz/b2djYyPPPPMMc+bMYe7cuWzZsgW32w2AxWIhICCg+7Hn\niuOCBQtYt24dAJs3b2b+/Pk9+8SF8BApaEL0srCwMCwWS/fHFouFsLAwoOuVvj/96U/s3r2bKVOm\n8LOf/eyK24UQQghPampqYu/evezZs4f9+/ezf/9+9u3bx5EjR1AqlZjN5u7Htra2Ul9fj8FguOR2\nlUrVfU82QHNz82XP+4c//AG1Ws3atWv54osvmDZtWve+4ODgC45fUVGB3W5n7NixOBwOtm7dSklJ\nCZMmTeqtL4MQN5UUNCF62W233cby5ctxOp20t7ezevVqpk2bRlFREU8++SQ2mw2tVsuwYcNQKBSX\n3S6EEEJ42rp165gwYcIFQwXVajVTpkzBZrNx4MABKioqcLvd/OxnP2P58uVMmzbtktuNRiMmk4mG\nhgacTidr16697HkbGhpITU1Fq9VSWFjIwYMHaW9vB2DGjBmsXLkSgBMnTnD33XfjdDpRKpXMnz+f\nX/ziF8yYMaN7eKQQ3kbuQROiBx5++GFUKlX3x7/85S95+OGHKS8vZ8GCBSgUCubOndt9X1lsbCwL\nFy5Eo9Hg5+fHc889R2pq6iW3CyGEEJ62atUqHn300Yu2z5o1i7/85S/8/Oc/59FHH0WlUpGZmclj\njz2GTqe77PYlS5awePFioqOjWbRoEQUFBZc87+OPP84zzzzDihUrGDNmDM888wz//d//TVZWFj/6\n0Y945plnmDFjBn5+fvzud79Dr9cDXcMc3377bRneKLyawn1uQK8QQgghhBBerL6+nrvuuott27Zd\n8AKqEN5EhjgKIYQQQogB4U9/+hP333+/lDPh1aSgCSGEEEIIr1ZfX8/MmTOpr6/n8ccf93QcIXpE\nhjgKIYQQQgghRD8hk4QIIYQQPdTW1sYzzzxDU1MTdrudJ554AqPRyPPPPw9AWloaL7zwgmdDCiGE\n8ApS0IQQQogeWrlyJYMHD+bpp5+mtraWRx99FKPRyLPPPktWVhZPP/0027dvv2AtJyGEEOJSbnpB\nM5laenwMg8EXs7m9F9LcXJL75vHGzOCdub0xM3hnbm/MbDQGeDrCTWEwGCgqKgK6Ft8NDg6msrKS\nrKwsAKZPn87u3buvWtDk30jvyu2NmcE7c3tjZvDO3N6YGbwv95X+ffTKK2hqtXfOzCO5bx5vzAze\nmdsbM4N35vbGzLeKBQsWsGLFCmbNmkVzczOvvvoqP//5z7v3h4aGYjKZrnocg8G3V/6evbUYe2Nu\nb8wM3pnbGzODd+b2xszgvbm/ySsLmhBCCNGfrF69mujoaN58800KCwt54oknCAj4+heFa52Pqzde\n/TUaA3rlStzN5o25vTEzeGdub8wM3pnbGzOD9+UecFfQhBBCiP7kwIEDTJkyBYD09HQ6OztxOBzd\n+2trawkPD/dUPCGEEF5E1kETQggheighIYH8/HwAKisr8fPzIykpif379wOwceNGpk6d6smIQggh\nvIRcQRNCCCF6aOnSpTz77LM89NBDOBwOnn/+eYxGI8899xwul4vhw4czadIkT8cUQgjhBaSgCSGE\nED3k5+fHH//4x4u2f/DBBx5II4QQwpvJEEchhBBCCCGE6CekoAkhhBBCCCFEPyEFTQghbiHbtm25\npsf98Y8vUVVV2cdphBBCCPFNUtCEEOIWUV1dxebNG67psT/84dNER8f0cSIhhBBCfJPXTRJi7XSw\nNqeUkUkh6DQqT8cRQgiv8fvf/y8FBceYOnUss2fPo7q6iv/7v7/wm9/8HJOpDqvVyuOPf4fJk6fy\n/e9/h3//9/9k69YttLW1cuZMGZWVFTz55NNMnDjZ05+KEEII0efcbjdtHQ7MLZ1YWjuxtHRiabMx\ndFAIidGBfXZerytoBWVm/rbqCPffnsKsMXGejiOEEF7j/vsfZsWKfzJ4cBJnzpzmL395A7O5kXHj\nJjBv3kIqKyv46U//i8mTL1yvq66ult/97k/k5u5i9epPpaAJIYS4rPYOB6drmjFZrOi0Kny0anx0\navRaFT66r99Xqzw7kM/a6egqXa227vJlPvfxuULW2onD6b7ouWdqW3jirsw+y+Z1BS0+wh+AgtNm\nKWhCCK/1zy9PsK+w7rqeo1IpcF7iH4pzxqaHc9+M5Gs6VkbGUAACAgIpKDjGmjUrUCiUNDc3XfTY\nrKwRAISHh9Pa2npdmYUQQgxcDqeL8rpWSquaOVXd9ae6of2anqtWKfHRdRU4ve68IveNbTqtCtzg\ncrtxudzdb50uNy5311Uup8uNXq+hta0Ttwuc33isy+3G6XTT0m7rLmQdNudlsykVCoL8tcSF+xPs\nryM4QNf11l+LwV9HcmxQb30JL/216dOj94GwIB+iQv0oKjfjdLlQKeU2OiGEuF4ajQaATZu+oLm5\nmVdeeYPm5ma+9a2HL3qsSvX1cHK3+/IFUQghxMDldrupM1sprW7uLmRnalsuuMKk16rISDCQGB1I\nZIgvdocLq82BtdNJR6cDq81BR6fz6202B9ZOB03tNjqvUJh6k7+PBmOwT3fhCvbXYThXwAK6Pg70\n1aJUKm5KnkvxuoIGkJUSxobcMspqWvt0/KcQQvSV+2YkX/PVrnOMxgBMppYbPqdSqcTpvPAfQIvF\nQlRUNEqlku3bv8Rut9/w8YUQQgwcTW02Tp1Xxk5XN9PW4ejer1IqiA33JzEqkMToQAZHBRIZ6otS\ncWPFxuVy02H7urRZbV2lrsPmRKEApVKBUqHoenvufQWolEoUSggN8aPJYkWlVKBQntt34XP89Bo0\n6v5/cccrC9rwZCMbcssoKGuUgiaEENcoIWEwRUWFREVFExwcDMBtt83gv/7r3zl+/CgLFtxJeHg4\nb7/9uoeTCiGEuNnMLZ0cLDFxqraVwlONNDR3XLA/3OBDZmIog88WsvgIfzTq3puwT6lU4KtX46u/\nsXpiNAZgusHn9jde+VkMSw4FoLDMzIKJgzwbRgghvITBYGDFinUXbIuKiubddz/q/nj27HkAPPbY\ntwFITPz6Kl9iYjIvv/y3m5BUCCHEzVDb2M6BYhN5xSZKq5q7t/v7aMhKCiUxKpDBZ6+O+ftoPJj0\n1uKVBc0QoCfG6EdJRRN2h8srLlUKIYQQQgjhSW63mzO1rRwoNnGg2ERlfRsACgWkxwczKtXI9HEJ\nKJ1OFDc4VFH0nFcWNICMBAOVpjZKq5pIizd4Oo4QQgghhBD9jsvl5kRlE3lFJg6WmKhv6hq6qFYp\nGZEcxsjUMEYkhxHgqwXAGOrXo/udRc95dUHbvL+CgjKzFDQhhBBCCCHOsjtcFJSZOVBs4lCJieb2\nrgmg9FoV44dEMCrVyLDBIfjovLYKDGhe+7eSFmdAoYDjZWYWT73644UQQgghhBioOmwOjpQ2cqDY\nxOGT9Vg7u2btDfDVkD08mlGpRjISDHJrkBfw2oLmq1czKDKQU1XNdNgc6LVe+6kIIYQQQghxVQ6n\nC0tLJ/VNHTQ0d9DQ1EF9cwf1FisnKptxOF0AhAXpmZrVVcqSY4I8uqaXuH5e3WoyEgycqm6mpKKJ\nzMRQT8cRQgghhBDihtkdThqaO6lvstJwfgk7+765pRO3+9LPjQnzY1SqkVGpRuIj/GWSDy/m3QVt\nkIH1uWUUnDZLQRNCiF5yzz138N57H+Pr6+vpKEIIMWC4XG6a2myYWzoxt3TQ2NKJubmThuavC1hz\nm+2Sz1UoINhfR3JMEKFBekID9YQG6Qk7+zYkUI9O03trkgnP8uqClhwThFqloKDM7OkoQgghhBDi\nFuVwurC0dp4tX500Nne9bbc5qalvpbGlk6ZWG67LXP5SKRWEBuqJSTB8Xb7OK2KGAB1qldw7dqvw\n6oKm06hIig6iuNxCq9UuC+gJIcQVPP74g/z61y8RGRlJTU01P/7x0xiN4VitVjo6Ovi3f/sRQ4YM\n83RMIYTotzrtTo6faqSgzExDc9dVMEtLJ81tNi4z8hCVUkGwv47EmEBCAnQYAnQYAvSEBOgIDtAR\nGqgnyF+LUoYkirOuqaC9+OKL5OXl4XA4+O53v8vs2bO7982YMYPIyEhUqq7Lqr/73e+IiIjom7SX\nkDHIQFG5haIzZkanhd+08wohhLfJzp7Ozp07WLLkPnJytpOdPZ2kpBSys28jL28f//jHu/zqV7/1\ndEwhhOhXmttsHDpRz6GSeo6fbsTmcHXv06iVGAJ0pIUGdxcvQ3cJ05EyKBRbh03Kl7guVy1oubm5\nlJSU8PHHH2M2m7nrrrsuKGgAr7/+On5+fn0W8koyEgysyjlFQZkUNCGE91hx4jMO1h25rueolAqc\nrsu9RgsjwzO5O3nhZfdnZ0/n5Zf/jyVL7uOrr7bz/e//Gx999Hc+/PDv2O129Hr9deURQoiBqrqh\njUMl9RwsqedkZVP31bHoMD9GpoSRlRRKVKgffnr1FSfjMATqMXXab05oMWBctaCNHTuWrKwsAAID\nA7FarTidzu4rZp42OCoQnUYl96EJIcRVJCYm0dBgora2hpaWFnJythEWFs5Pf/oLCguP8/LL/+fp\niEII4REul5uTVU3dpaymsR3ompwjJTaIESlGRqaGEWGQyZNE37tqQVOpVN0zeS1fvpzs7OyLytnP\nfvYzKisrGT16NE8//fRNndZTrVKSGhfMkdIGzC2dGAJ0N+3cQghxo+5OXnjFq12XYjQGYDK19Oi8\nEydO4W9/+wtTp07DYjGTlJQCwPbtW3E4HD06thBCeJNOu5Pjpxs5WFLP4RP1NLd3XenSapSMTAlj\nZIqRrORQAn21Hk4qbjXXPEnI5s2bWb58OW+99dYF25988kmmTp1KUFAQTzzxBBs2bGDu3LmXPY7B\n4Ita3fOrb0ZjQPf7Y4ZEcqS0gSqzldTEsB4fuy+dn9ubeGNub8wM3pnbGzODd+buaeZFixawbNky\n1qxZQ3t7O8888ww7d27jwQcfZOvWTezYsRGVSklYmL/Hhq4LIURfaW6zkX+y636yY6e+vp8s0E9L\n9vAoRqQYGZJgQCtT1gsPuqaClpOTw2uvvcYbb7xBQMCFvxwsXry4+/3s7GyKi4uvWNDM5vYbjPq1\nb76KHB/WdYVvz9FqhsYH9/j4faU3Xv32BG/M7Y2ZwTtze2Nm8M7cvZE5Kmow27fvASAoCN5775/d\n+z76aBUA2dmzaW930d7e86+PN5ZgIUT/4na7ae+wY27ppMPmoMPmpKPz7Fubs3ub9YLtF+63dna9\nbe9wdN9PFhXqy4izV8oSowNlIg/Rb1y1oLW0tPDiiy/yzjvvEBwcfNG+p556ildffRWtVsu+ffuY\nM2dOn4W9nLgIf/z0agpOm3G73bJyuhBCCCHEAHCqupn3viiirPbGXjBSKhTotSr0OhVB/jriIwIY\nlhjCyBQjkSFyP5non65a0NavX4/ZbOapp57q3jZ+/HjS0tKYNWsW2dnZLF26FJ1Ox5AhQ6549ayv\nKBUK0uMN5BWbMDV1EB7sc9MzCCGEEEKI3mHtdLAyp5QteRW43ZCVHIZeo+wqW1r11291KvRaFT4X\nbVPjo1WhUSvlhXvhda5a0JYuXcrSpUsvu//RRx/l0Ucf7dVQNyI9oaugFZaZpaAJIYQQQnipQyX1\nvL+piMbmTiIMPjwyN53sMfFeNyxd9B8djk52V++j3d6OUqFCqVCgVChRKZRnPz73vvLr95WX2a5Q\nEesfjV7ddxMTXvMkIf3dkEEGAI6fbiR7eLSH0wghhBBCiOthae3kg03F7C8yoVIqWDhpEHdMSkDT\nC5PLiVvX0foCPi5eRWNH7y3JNcKYybczH+61433TgClokSG+BPlrKSyT+9CEEEIIIbyFy+1mx6Eq\nPtl2Emung6SYQB6dm06s0d/T0YQXa7a1sLx4DXl1+SgVSmYnTCcjJBWX24XT7cLlduJyu3C53bjc\nzrPbXOftP/e+86LtmWEZfZp9wBQ0hUJBRoKB3GO1VNa3yTe1EEIIIUQ/V1nfxrtfFHKiogkfnYqH\nZ6cybWSMzKh4HY7WF7D65OcAjAzPZFT4cCL9wj2cynNcbhe7q/ex8sR6rA4rgwLjeSB9CTH+UZ6O\nds0GTEEDugtaQZlZCpoQQgghRD9ldzj5bFcZ63PLcLrcjE4z8sDtqRgC+u6+noHG3GFheckaDpmO\ndt8jte7UJtad2kSMfxSjwoczOnw4Rt9QT0e9aWra6vig8FNONp1Cr9JxX+pipsZMQKlQejradRlw\nBQ2gsMzMrDFxHk4jhBBCCCG+qeiMmXe+KKK2sR1DgI6HZqUyMtXo6Vhew+lysr1iJ5+d2kin00Zi\n0CDuT7sbgz6YI/XHOVCXz/GGYtaWfsHa0i+ID4hhVPhwRoVnEeoT4un4fcLucvDJ0c9YefwLHG4n\nw43DuC91EcG6IE9HuyEDqqCFBflgDNZTeMaC0+VCpfSutiyEEEIIMVC1Wu18svUEOYerUQAzR8dy\nd3YiProB9etonzrVVMaHRSuobK3GT+3LPemLmBA1uvsK0bjIUYyLHEW73Up+/TEO1OVT2FjCmZZK\nVp1cz6DAeEaHZzEyPAuDPvgqZ/MOJyyn+KDwU2rb6wjWBXFf6iKGG4d5OlaPDLjviIyEEHbkV3Gm\ntpXBUYGejiOEEOIW8Mknn7BmzZruj48ePcqHH37I888/D0BaWhovvPCCh9IJ4Vlut5u9BXV8uLmY\n5nY7sUZ/Hp2XRlK0d17d8IR2ezurT37Ozqq9uHEzMWosi5Pm46/1u+TjfTU+TIwaw8SoMbTa28g3\nHeVA7WGKLSc53XyGT098RmLQIEaHD2dkeCZBOu/7nbnd3s6qk+vZWbUXBQrmJt/G7dEz8FHrPR2t\nxwZgQTOwI7+KgjKzFDQhhBA3xb333su9994LwN69e/n888/51a9+xbPPPktWVhZPP/0027dvZ9q0\naR5OKsTNVW+x8t7GIo6WNqJRK7nntiRmj41DrfLuUU42pw2tStvn53G73eyrPciKks9osbcS5RfB\nsrS7SQ4efM3H8Nf4MTl6PJOjx9Nia+WQ6QgHag9TYimltOk0y0vWkBw8mFFny5qRgD78jHrO7XZz\noC6fT0rW0GJrJdovkgfSlzAuediAWStvwBW09LP3oRWcbmT+hAQPpxFCCHGreeWVV/jNb37DQw89\nRFZWFgDTp09n9+7dUtDELaPW3M6e47Wszy3DZncxZJCBR+akEW7wve5jtdrbcLZYUeHTB0mvnd3l\n4GDdYXIqd1PaVEaEr5GhoekMC80gKXgQamXv/lpd01bHx0UrKbacRKPUsChpHjPipvboPAFaf6bG\nTGRqzESaOls4aDrMgdp8SiyllFhK+WfxKmKDosCl+MYizapLLNh87n3V2fcVFyz6rFVqCPUJxegT\ngtE3jEBtQI8n62iwmvm4eCXHGgrRKNUsSpzHzPhsVMqBtVbegCtoQX5aYox+lFQ0YXe40Ki9+xUa\nIYQQ3uPw4cNERUWhUqkIDPx6FEdoaCgmk8mDyYToWy63m1PVzRwqqedgST1V9W0A+PtoeGROGhOH\nRt7QGrU2p52X9r9CnbWe+IAYxkeOYUzEiMsO7esLjR1mcipz2VW1l1Z7GwoUxAfEUtNWy5flOXxZ\nnoNepSM9JJVhoekMCU0nSHfjV6FsTjsbyr5kU9k2nG4nw0IzuC91Ua9P8BGkC+C22MncFjsZc4eF\ng2evrNW21+F0OS9Y96s3aJRqwnxCCfMJxdj9J4wwn1BC9MFXLFnnJkZZW7oBm8tOuiGFZWl3D9gZ\nKgdcQQPIiDdQaWqjtKqJtHiDp+MIIYS4RSxfvpy77rrrou1ut/uanm8w+KJW9/yVYKOxfw9Ruhxv\nzO2NmaF3ctvsTvJLTOw5VsPeYzWYWzoB0KqVjBsSyfhhkUzKjMLf98aHAv4jfyV11nqiAsKpaK3m\nTMlqVpz8jFFRw5g2aAKjooahVvX+r7Mut4sjtYVsKNlOXvUR3G43/lo/7ki7nVnJ2UT6G7E57Ryv\nK+FA9REOVh3lkOkIh0xHAEgyJDAyeiijlJkkhsVf85WjQ9XHeDPvI2rb6gn1MfDYqPsYGzP8hsrt\n9TASQGpcHEuZf9E+t9uN2+3GeW4xZ5frMu93vXW6uhaBbrd3UNdWT02riZpWE7Vn31a31V50DpVC\nSZhfKJH+RiL8w4j0Dyfy7NsORydvHPyQUvMZArR+fGfkg0xNGHfJr4m3fj9+08AsaAkGNudVUFBm\nloImhBDiptmzZw8/+clPUCgUWCyW7u21tbWEh1994Vizub3HGYzGAK+8D8Mbc3tjZuhZ7larnfwT\n9RwqqefoqUY67U6g60rZ5MxIRqYYGTooBJ2264UGa1sn1rbOGzpXeUsVa4s2E6o38L+zn6WipoH9\ntQfZU5PHvsp89lXm46fxZUzECMZHjiY+ILbHRabd3k5u9X5yKnOps9YDkBAQx9TYiYwOH45WpQEr\nmKxdX78YdRwxcXEsjJ1HbbuJow0FHKsv5ITlFCfNZSw/tp4AjT9DQtMYFpZBRkgKPuqLh2paOptY\nXrKWg3WHUSqUzIzLZv7gWejVOurrW3v0OV2va///QwGoANXZ/4LmvL0BSogIiCbzvM7kdrtps7dj\nstZjsjZgsjZQb23A1N71Nr/1+GXPNj5yNHcnL8Rf63fJr4m3fT9eqUwOyIKWFh+MQgEFZWYWT/V0\nGiGEELeC2tpa/Pz80Gq7rhYkJiayf/9+xowZw8aNG3n44Yc9nFCIG1Nnbu8eulhS0YTr7BXhCIMP\nI1OMjEgJIzkmCKWy967yuNwuPihcjsvt4v60JejVOoJ0AcyMz2ZmfDYVLVVdRa32INsrdrG9YheR\nvuGMjxrNuMhR173+1ZmWCnIqdrOv9hB2lx21Us2EyDFkx04kIfDqa+sqFAoi/cKJ9Avn9vhpWB0d\nFDaWcKLtBHmVR9hTk8eemjyUCiVJQYMYFpbBsNB0jD5h7KjczWelG+hwdjI4MIH70+8mxj/qRr90\n/ZpCocBf64e/1o/BQRfPFdHh6MBkbcRkre8ubu2OdqbGTCQ9JMUDiT3D6wqa1WFlVcEuRgePQn+Z\naTR99RoGRQZQWtVMp83Z/SqOEEII0VdMJhMhIV/fI/Lss8/y3HPP4XK5GD58OJMmTfJgOiGuncvt\n5nR1CwdLTBwqqafy7P1kCiAxOpARKWGMTDESFerbZ0PvtpV/xZmWCsZGjCIjNPWi/bEB0cQGRLM4\naT4FjcXsqcnjcP1xVp/8nDUnvyDNkMz4qNGMMA677GyLdqedA2cn/TjVfAaAMH0IU2MnMiFqDP6a\nG7/PzUetZ2R4JrONk6ita6K8pZKjDYUcqy/snpBj5Yl16FV6Opwd+Kp9eCBtCROjx/Z4Ig1vplfr\niQuIJi4g2tNRPMrrCtpJy2k+OLyKxkEt3JE457KPy0gI4VR1C8UVFjITB+YNhEIIIfqPYcOG8cYb\nb3R/nJyczAcffODBREJcO5fLTUmFhf1FJvKK6rC02gBQq5RkJYUyMiWMEclhBPnr+jxLg7WRtaUb\n8NP4siRl4RUfq1Kquq5GhWXQbm/nQN1h9tTkUWguodBcwkcqLSPDs5gQOZqk4MEoFUoarI3kVOay\nu3pf96Qfw0LTyY6dREZIaq8XJKVCSUJgHAmBcSwYPItmWwvHGoo4Vl9AaVMZw41DuSt5AQFa/149\nr/BeXlfQUg1J+Gl82FW1l/mDbr/sjC8ZCQbW55ZRUGaWgiaEEEII8Q0ul5vicgv7iuo4UGSiqa2r\nlPnp1UweFsmIFCPDBofc1JGIes1hAAAgAElEQVRIbrebj4pXYnPZWZZ293WVFl+NL1NiJjAlZgJ1\n7Sb21hxgT80Bcqv3k1u9n1C9gXBfI4WNJbhx46fxZVb8bUyJmUBYL8+QeCWB2oDuRaSFuBSvK2ha\nlZZpgyawvmQr+fXHGBWedcnHJccGoVYpKCgz3+SEQgghhBD9k9PlIr/YxOa9ZRwoqqO53Q50TfKR\nPTyKMenhpMcbPLaQdF7tIY43FJFuSGFc5KgbPk64r5GFiXOYP3gWJyyn2FOTx8G6wzQ0mhkUGE92\nzERGhWehUWmufjAhbjKvK2gAtydPZX3JVnIqcy9b0HQaFUnRQRSXW2i12vH3kW9AIYQQQtx6nC4X\nhWUW9hfVcaDYRMvZUhbgq+G2EdGMTg8nPT4YldKz9z612tv4pGQNGqWG+9Pv7pX725QKJamGJFIN\nSdyXuphWW2uvrycmRG/zyoIWGxhFSnAixeYT1LbVEeF36amLMxIMFJVbKDpjYXSa8SanFEIIIYTw\nDIfTRWGZ+Wwpq6fV2lXKAn01zJs0iGHxwaT2g1J2vpUl62i1t7E4aT5hPr1/e4pOpUUn5Ux4Aa8s\naABTYyZQYinlq6o9LEm545KPSU8wwFenKCwzS0ETQgghxIDmcLo4ftrM/sI6DpaYaOtwABDkp2XG\nqBjGpIWTGhdMRERgv1svqrCxhNya/cT5RzMjTtZIErc2ry1ow43DCND4k1u9nzsS53YtHPgNidGB\n6DQqjpc1eiChEEIIIcSVudwu2uztPZrBz+12syO/iuXbTnaXsmB/LTNHxzI2PbzX1yjrbTannQ+L\nVqBAwQPp91x2AjghbhVeW9DUSjUTo8eysWwrB+rymXCJmXDUKiUpcUEcLW3E0tpJ8E2YGlYIIYQQ\n4lp9VLSCXVX7WJq2mKkxE6/7+R02B+99UUTu8Vp8dWpuH9NVypJiglD20RplvW39qU3UWxuYGZdN\nfGCsp+MI4XH9Z+DxDZgSPR4FCr6qzL3sYzISDAAUymyOQgghhOhH6trr2VW1DzduPipayeqTn+Ny\nu675+eV1rfz8nf3kHq8lMTqQ5x8fywO3p5ISG+w15ay8pYot5TsI1RtYkDjb03GE6Be8uqCF+oQw\nJDSNU81nKG+puuRjhiR03Qx6XAqaEEIIIfqRjWVbceNmUdI8wn3C2Fi2lXePf4Td5bji884Nafzl\ne/upaWxn9tg4/uvBUYQF+dyk5L3D5XbxQeFyXG4X96ctQafSejqSEP2CVxc06JosBOCryt2X3B8X\n7o+fXi1X0IQQQgjRb9RbG9lTk0ekbzi3x0/j6dFPkBiUwP7aQ7xy6A3a7dZLPq/D5uCNz47zzueF\naFRKfnB3Jstmpnhs3bKe2Fb+FWdaKhgbMYqM0FRPxxGi3/C+7+ZvGBqajkEXzN7ag1gdHRftVyoV\npMUbqG/qwGS59A87IYQQQoibaWPZVlxuF3MHzUSpUOKv9eMHI77DCGMmJZZSfn/gLzR2XPjicsXZ\nIY27j9UyOCqQ5x8by8hU75ylusHayNrSDfhpfFmSstDTcYToV7y+oCkVSqbEjMfmtLGv5sAlH3Pu\nPrQCuYomhBBCCA9r7DCTW72fcN8wRkcM796uVWn4/4Y9yPS4KVS31fK7/S9T3lLVPaTxF+cNafzx\nQ6MIC/auIY3nuN1d99zZXHaWJN/RoxkshRiIvL6gAUyMGodSoSSnMhe3233RfiloQgghhOgvNpVt\nw+l2Mjeh6+rZ+ZQKJfek3MmSlDtotrXyhwN/4fefbxoQQxrPyas9xPHGItINKYyLHOXpOEL0O977\n3X2eIF0Aw43DqGqrobSp7KL9UaG+BPlrKSgzX7LACSGEEELcDJbOJnZV7SVMH8KYiBGXfdyMuKks\njr+HTruDk7otRCTXe/WQxnNa7W18UrIGjVLD/el3o/CS2SaFuJkGREEDyD47WUjOJabcVygUZCQY\naG6zUVXfdrOjCSGEEEIAXVfPHG4ncwbNuOyCzG63m5z8KpavaqOzcCwatDSH7GdPY47Xv9C8smQd\nrfY2FibOJswn1NNxhOiXBkxBSwlOIsLXyMG6fFptF5ewjHgZ5iiEEEIIz2nqbGZn1R5C9IbLDu3r\nmqWxgLc/L0StUvKvs7N5duKThOlDWH96M+8XfILT5bzJyXtHYWMJuTX7iQuIYXrsFE/HEaLfGjAF\nTaFQMCVmAg63k9ya/Rftl/vQhBBCCOFJm89sx+5yMDthOmql+qL9FaZWfvHufnYfq2FwVAA/e2ws\no1KNRPga+Y8x3ychII7cmv38Jf+tS85c3Z/ZnHY+LFqBAgUPpC+57NVDIcQAKmgA4yNHo1GqyanM\nxeV2XbAvLNgHY7CeojMWXC7vHh4ghBBCCO/SYmslpzKXYF0QE6LGXLDP7XaTc7iKX767n+qGdmaN\niePHD43GeN4sjQFaf3446rtkhmVQaC7hDwdexdLZdLM/jRu2/tQm6q0NzIibSnxArKfjCNGvDaiC\n5qfxZXT4COqtDRSZT1y0PyPBQHung7LaFg+kE0IIIcStasuZHdhddmYnTEdz3tUza6eDN9cV8Pb6\nQlQqJd+/O5P7b7/0LI06lZbvZD7K1JiJVLZW89v9L1PZWn0zP40bUt5SxZbyHYTqDSxInO3pOEL0\newOqoAFMucJkIelnhzkWyjBHIYQQQtwkrbY2tlfuIkgbwKSosTicLg6frOeva47xb3/+il1Hu4Y0\nPn92SOOVKBVKlqYuZnHSfCydTfw+71WKGi9+Ubq/cLldfFC4HJfbxf1pS9CptJ6OJES/d/EAaC83\nKDCOOP9ojtQfx9LZRLAuqHtfRkIIAMfLzMybkOCpiEIIIYS4hWwtz8HmtDE59Db++eUp9hbU0tJu\nByDc4MOUzCjmjo+/5rXNFAoFsxJuw6AL4u8F/+SV/Dd5KOPefrmm2LbyrzjTUsHYiFFkhKZ6Oo4Q\nXmHAFbRzk4V8WLSCnVV7WTB4Vve+ID8tMWF+lJRbcDhdXr3IoxBCCCH6vzJTA5vKclA4dKxf7wZX\nBQG+GmaOjmXi0EgGRwXc8FpgYyJHEqQL5K9H3uPd4x9Rb21gVHgWPmoffNQ+aJRqj64z1mBtZG3p\nBvw0vixJWeixHEJ4mwFX0ADGRIxk5Yl17Kray9yEC9cZSU8wUFnfRmlVM6lxwR5MKYQQQoiBqKXd\nxr7COnYfq6GMPDQxdlw16YxPj2bi0AiGDArptReJUwxJPD36X3nl0JusO7WJdac2de9TKVT4qPX4\nni1sPmo9Pmo9hoBAFHZV1zbNuf367sfoVXrcuHG47NhdDuwuB47ut+dvs39jnwP7uW1OB2daKrC5\n7CxLu5sArX+vfL5C3AoGZEHTq3WMixzFjsrdHG0oYLhxWPe+IQkGtuRVcPx0oxQ0IYQQQvQKm93J\noRP15B6r5UhpA06XG4Xaju+IM2gUPjx3zzKCfX375NxRfhH8aMz3yanMpcXWgtXRcfaPlfazb82d\nTdhdXcMqMfVJjEsaYczsl0MvhejPBmRBg67JQnZU7ianMveCgpYWH4xCcXaikKkeDCiEEEIIr+Z0\nuSk43cjuY7XsL6qjw9a1gHR8uD8ThkbSHlzAlko7cxNv77Nydk6QLpCFV5kh0e5y0OHoQB+opKqu\ngXaHtavI2a20O6x0ODrOFroOVAolapUajUKNWqlGo1SjUWq6tinVqJUaNAoVapXm7L6z27rf7/oT\npA306DBLIbzRgC1oMf5RJAYNoqCxGFN7A0bfUAB89RoSIgI4WdVMp82JTisLJQohhBDi2tkdLjbs\nPcP2/CoamroWjA4N1DFjVCwThkYQa/Snw9HBT3e9iZ/al+yYiR5O3EWjVKPR+mMMCEDT0beFUQhx\n4wZsQQOYGjOB0qbT7Kzaw+Lk+d3bMwYZOF3TQkmFhWGJoR5MKIQQQghvcux0I+9vKKLWbMVPryZ7\neBQTh0aSEheM8rwrRTsqdtPusLJw8Bz0ar0HEwshvM2AnsZwpDETP40vu6v3YXc5urdnnF0PrUDW\nQxNCCCHENWhq7eRva47x0keHqLNYuX1MLG/9dDb/Mi+DtHjDBeWs02ljS/kOfNR6boub5MHUQghv\nNKCvoGlUGiZGjWXzme0cqjvC2MiRAKTEBqNSKqSgCSGEEOKKXC432w5V8un2UqydDgZHBfDInHQS\nIgPw1Wtoa+m46Dk5lbtptbcxf9Dt+Kh9PJBaCOHNBnRBA5gSPYHNZ7aTU7m7u6DpNCqSYoIoKbfQ\n1mHHT6/xcEohhBBC9DdlNS28t6GQU9Ut+OjUPDw7lWkjYlAqLz/phc1pY/OZ7ehVOqbHTbmJaYUQ\nA8WAL2hG31AyQlIpaCymqrWGaP9IoGuYY3G5haIzFkalGj2cUgghhBD9hbXTwcodpWw5UIHbDROG\nRLB0RjJB/rqrPndn1V5abK3MTZiBr0Ym4hBCXL8BfQ/aOVNjJgCQU5nbve3cfWiHTzZ4JJMQQggh\nro3daedQ3RHa7O19eh63282+wjqefT2XzXkVhAf78B/LRvCdO4deUzmzO+1sKtuKVqVlerys5SOE\nuDED/goawLDQDIJ1QeytyWNR0jz0ah3JMUEYAnTsLahl2cxk9Npb4kshhBBCeA23201eXT5rTn5O\nQ4eZGP8onhr5PXw1vX9fV525nfc3FXO0tBG1SsniKYOZNyEejfral+PZVb2PJlsLs+Jvw1/j1+sZ\nhRC3hmu6gvbiiy+ydOlSlixZwsaNGy/Yt2vXLu655x6WLl3KK6+80iche0qlVDEpehwdzk7yag8B\noFQqmJoVRYfNyb7COg8nFEIIIcT5TlhO8du8l3n72Ac0dTYzODCBytZqXj38NjanrdfOY3e4WLvz\nFD99cy9HSxsZOsjAL741jjunDL6ucmZ3OdhYthWtUsPM+OxeyyeEuPVc9bJRbm4uJSUlfPzxx5jN\nZu666y5mz/56pfpf/vKXvPnmm0RERPDQQw8xZ84ckpOT+zT0jZgcPY4vTm8hp3I3k6LHoVAomJIV\nxdqdp9mRX8XUrGhPRxRCCCFuebXtJlaf/Jx801EARocP586kuYToDbxz7EPy6vJ5/ejf+W7mo6iV\nPRv9UnC6kb9vLKamsZ0gPy33L0hhbHo4CsXlJwG5nNzq/Vg6m5gZl02A1r9HuYQQt7ar/mQbO3Ys\nWVlZAAQGBmK1WnE6nahUKsrLywkKCiIqKgqAadOmsXv37n5Z0IJ1QWSGDSHfdJSylnIGBcYTFuTD\n0MQQjpY2UmlqJcYoP1CFEEIIT2jubOWfxavJqdyNy+0iMWgQdycvYHBQQvdjHhmyFKujg+MNRbx3\n/GP+Zej9KBXXfzt9U5uNf35Zwu5jtSgUMHN0LHdNTcRXf2OFz+FysOH0l2iUambGT7uhYwghxDlX\n/UmkUqnw9e2ahWj58uVkZ2ejUnVd8jeZTISEhHQ/NiQkhPLy8j6K2nNToyeQbzpKTkUug4bEA5Cd\nFc3R0kZ25Fdz/+0pHk4ohBDCW61Zs4Y33ngDtVrNk08+SVpaGv/5n/+J0+nEaDTy29/+Fq1W6+mY\n/Y7daWdbxU42nPkSq70Do08oi5PmM9w47KIrWWqlmm9lPszLh14nry4fX40vS1MXX/MVL7fbTc7h\nav755QnaOx0kRAbwyJw0BkcF9uhz2FtzAHOnhdtiJxOkC+jRsYQQ4ppfKtq8eTPLly/nrbfe6tEJ\nDQZf1NcxpvtyjMbr/wEYGjaST04ayTPl852gZfhr/bjd4McHm0vIPV7D9+4ZjlbT82xXciO5+wNv\nzO2NmcE7c3tjZvDO3N6Y+VZgNpt55ZVX+PTTT2lvb+fPf/4zGzZs4IEHHmDevHn8/ve/Z/ny5Tzw\nwAOejtpvuNwu8mrzWX3yc8ydFvy1ftyTcidTYyZcceiiTqXl/2U9xh8OvEZO5W78NL7ckTjnqudz\nulx8sKmErQcr8dGpeHBWKtNHXnlNs2vhdDn54vSXqBUqZiXc1qNjCSEEXGNBy8nJ4bXXXuONN94g\nIODrXw7Cw8Opr6/v/ri2tpbw8PArHsts7vkUuUZjACZTyw09d2LEWFadXM+6o9uZEdc1Be7EoRF8\nvucMG3aVMmFIZI/zXU5PcnuSN+b2xszgnbm9MTN4Z25vzXwr2L17NxMnTsTf3x9/f39+8YtfMGPG\nDF544QUApk+fzltvvSUF7awS80lWnFjHmZYK1AoVt8dP48HRd9Le5Lym5/tqfPn+iG/x+wOv8sXp\nLfipfZhxhYk5rJ0OXl19lKOljcQa/Xnq3ixCAvW98rnsqz1IQ0cj2TETCdYF9coxhRC3tqsO3G5p\naeHFF1/kr3/9K8HBwRfsi42NpbW1lYqKChwOB1u3bmXy5Ml9FrY3TIwai1qh4qvKXNxuNwBTh3dN\nEJKTX+3JaEIIIbxURUUFHR0dfO973+OBBx5g9+7dWK3W7iGNoaGhmEwmD6f0vNq2Ov56+F3+7+Bf\nOdNSwZiIETw34UfclbwAP+31LeocpAvkByO+TZA2gE9PfEZu9f5LPq6hqYPfvJ/H0dJGspJC+fFD\no3qtnLlcLjac/hKVXD0TQvSiq15BW79+PWazmaeeeqp72/jx40lLS2PWrFk8//zzPP300wDMnz+f\nwYMH913aXuCv9WNkeBb7ag9SYjlJqiGZyBBf0uKCKSgzU2duJ9xwff9ICCGEEBaLhZdffpmqqioe\neeSR7hcBgQvevxJP3gbQl5o6mvnk2Do2n/wKl9tFhjGZh4cvITl00AWPu97cRgL4aeAP+dnW3/OP\nwuVEhoYwNmZ49/6ScjO/fj8Pc0snCycP5luLhqFSXf+kIpfzVdle6qz13J44hbS4+F477s3Q3/4f\nuRbemBm8M7c3Zgbvzf1NVy1oS5cuZenSpZfdP3bsWD7++ONeDdXXpsZMZF/tQXIqc0k1dM04mT0i\nmqJyCzvyq7nntiQPJxRCCOFNQkNDGTlyJGq1mvj4ePz8/FCpVHR0dKDX66/pFgDw/G0Avc3pcrLl\nzA42lH1Jh7OTcJ8wFifPJytsKAqX4oKcN5rbh0D+X+Zj/OnQ6/xh1xs8MfxxUg3JHCg28bc1x7A7\nXNx/ewqzxsTR2NjWa5+by+3i0+Ofo1QoyY6Y0m++5teiP/0/cq28MTN4Z25vzAzel/tKZbL3Xkby\nIolBCUT7RXLIdJSmzq6/yNGpRnx1anYeqcbhdHk4oRBCCG8yZcoUcnNzcblcmM1m2tvbmTRpEhs2\nbABg48aNTJ061cMpb76vqvawuvRzVEoV96Yu4ifjn77k7Iw9NTgoge9kdl21fO3wO3y8ex+vrDgC\nCvjBkixmjYnr1fMBHKw7QmVzDeMjRxPqE3L1JwghxDW6JQuaQqFgaswEXG4X2yt2AqDVqJg4LJKm\nNhuHTzZ4OKEQQghvEhERwZw5c7jvvvv49re/zU9+8hN+8IMfsGrVKh544AEsFguLFy/2dMyb7mh9\nAQDPjPkht8VORqXsu5mSM0JSeSRjGZ0OO9ubVxEQ0smPHxzNiJSwXj1Pg9XM8uI1vF/wT5QKJXMS\nZvTq8YUQ4sZWZBwAxkWOYkPZVjad2cYI4zDiA2PJHh7NlrwKduRXMSrV6OmIQgghvMiyZctYtmzZ\nBdvefvttD6XxPLvLwQlLKZF+EYT6GPr8fNZOBzu2u7G1DkE7+Bg+GXkEBE/oteNXtlazqWwbeXX5\nuNwuDLpgHhyxGKNvaK+dQwgh4Ba9ggagV+t5KONeXG4X7xz/CJvTTly4P4OjAjlS2kBjc4enIwoh\nhBBe61RTGTaXnQxDSp+fq3umxlONDA0cwYKEuTTbm/nzoddpsbXe8HHdbjfF5hO8cuhNfr33D+yr\nPUikbziPZCzlhYnPkD1ofC9+FkII0eWWvYIGXcMhboudzLaKnaw+uZ57UxcxbUQ0pz5v5qvD1dw5\npX/PSCmEEEL0V4WNJQCkhST36XlOVTfzp+WHaWqzMXNULMtuT0alVGJzd7DpzDZeOfQGPxz1XXzU\nPtd8TJfbxSHTUTaXbaespRyAlOBEbo+fxtDQ9F6/h04IIc53Sxc0gEVJ8yloLGFbxU6GhWUwLiOR\nD7eUkHO4ioWTBqFUyg9hIYQQ4noVNpagVChJCU7ss3PkFZl4fe2FMzWesyhpHm32dnZV7+W1w+/w\nxPBvoVVprng8m9POnpr9bD6zg3prAwoUjDBmcnv8NAYHedc0+kII73XLDnE8R6vS8C9DlqFUKHm/\n4BNcChvjM8JpaO7k2OlGT8cTQgghvE67vZ0zLRUMDkxAr+6dRaHP53a7+WLPGf6y8ggKheKSMzUq\nFAruT7+bkcZMTlhO8dax93G6nJc8Xpu9nS9Ob+G5Xb/ho6KVWDqbmBw9np9O+A++nfmwlDMhxE11\ny19BA4gPjGX+oFl8dmoDHxevYtrwhezIr2ZHfhWZiXLzrxBCCHE9iswnceMmvQ+GNzpdLv6xsZht\nh6oI9tfyw3uGkxB56fWElAoljw69H2t+B0fqC3i/8BMezrgPpaLr9enGDjNfluews2ovNqcNH7We\n2QnTuS12CkG6gbHgrRDC+0hBO2t2wm0cayhgf+0hhoVmEGv051BJPU1tNoL8tJ6OJ4QQQniNwsZi\nANJDUnv1uNZOB6+uOsrRU43Ehfvzw3uyCAm88hU6jVLNtzMf4c+HXmdvzQF81T5Mih7H5jPb2V97\nCJfbRbAuiAWDZzElenyfXPETQojrccsPcTxHpVTxyJBlaJUaPi5exbisQJwuN7uOVHs6mhBCCOFV\nCs0n8FHrSQiI7bVj1jdZ+fXZmRqzkkL5rwdHXbWcnaNX6/jX4Y8T5RfBtoqd/HrvH9hbc4BwXyMP\nZ9zHCxOf4fb4aVLOhBD9ghS084T7hnF3yh1YHVZOKHegUSvYkV+F2+32dDQhhBDCK9RbG6m3NpAa\nnNRrC1ObLFZ+9V4elaY2Zo6O5QdLMvHRXd8gID+NL98f8S1i/aNJDh7M97L+hf8e929MiBqDWikD\nioQQ/Yf8RPqGKdHjOVJ/nGMNhSQMDeFEfgjF5RbS4vt+kU0hhBDC250b3pgW0jvrnzldLv629hhN\nbTbunZ7EvPEJN3ysYF0QPx73VK/kEkKIviJX0L5BoVDwYPq9+Gl8qdEfQKFvZXt+ladjCSGEEF6h\n0HwCgPReKmhrd57mZGUzY9PDmTtOZlMUQgx8UtAuIUgXwANpS3C6HfimHmV/YS1tHXZPxxJCCCH6\nNZfbRXHjCQy6YMJ9wnp8vJIKC2t3nSYkUMcjc9NkgWghxC1BCtpljAjPZHzkaFx6C0SWsPtojacj\nCSGEEP1aeUslbY520kNSelym2jsc/G3NcQC+vXAIfvorLzIthBADhRS0K7g39U6CtcGoo0+ypfCI\nTBYihBBCXEFRY+8Nb3x/YxENzR0smJgg94ELIW4pUtCuwEftw78MXYoCsITspbiywdORhBBCiH6r\nwFwCQJqhZwtU7z5aQ+7xWhKjA7lz8uDeiCaEEF5DCtpVpBiSGB40DqW+nQ8LVns6jhBCCNEv2Zw2\nSi2niPWPJkDrf8PHMVms/H1jETqtiu/cMQS1Sn5VEULcWuSn3jV4dOQiFB2BmFRFHKg+5uk4Qggh\nRL9z0nIah9vZo+GN56bU77A5eWhWKuEG315MKIQQ3kEK2jXQqTRM9J+L26XgH4Wf0Gpr83QkIYQQ\nol8pMHetf9aTgnZuSv1xGeFMGhbZW9GEEMKrSEG7RvNGDMNRmUKHu50Piz6VCUOEEEKI8xQ2lqBW\nqkkKurF7xo6famDtrtOEBup4ZI5MqS+EuHVJQbtGIYF6MnxH42w2cMh0lL01BzwdSQghhOgXWmyt\nVLZWkxQ0CK3q+qfDb+9w8NI/8gD49h1D8ZUp9YUQtzApaNdh2vBY7KWZqNwa/lm8mgar2dORhBBC\nCI8rauyavfFGhze+v7GIOrOVBRMHkRoX3JvRhBDC60hBuw5ZSaEEaoJxlg+hw9nB3ws+xuV2eTqW\nEEII4VHnptdPN1x/QTs3pX5avIE7Jw/q5WRCCOF9pKBdB7VKyZSsKKw1kcRpkyixlPJleY6nYwkh\nhBAe43a7KWo8gZ/Gl9iA6Ot6bt15U+o//eBomVJfCCGQgnbdpmZFAQpc5ZkEaPxZe/ILKlurPR1L\nCCGE8Ii6dhPmTgtphmSUimv/tcLpcvH6eVPqR4X59WFKIYTwHlLQrlO4wZeMBAMnyjqYH3sHDreT\nd49/hN3l8HQ0IYQQ4qbrHt54nfefyZT6QghxaVLQbkD28K4hHDWnA5gcPY7K1mrWnvzCw6mEEEKI\nm6+w8frvPysut5ydUl8vU+oLIcQ3SEG7AaNSw/DTq9l1pJpFiQsI9wljS/kO9lTneTqaEEIIcdM4\nXU5KzCcx+oQS6hNyTc9p77Dz+trjAHz7jiEypb4QQnyDFLQboFGrmDQsiuZ2OwWlLXw361/wUev5\noHA5Jy2nPR1PCCGEuCnKWsrpcHaSHpJ6TY93u938fWMxDc0dLJQp9YUQ4pKkoN2g7OFRAOzIryLS\nL5xvDXsYF27+duRd6q2NHk4nhBBC9L2C7uGNydf0+N3HathzvJak6EDunDKoD5MJIYT3koJ2g2KM\n/iTFBHLsVCP1FivpISncl7qIVnsbrx1+G6ujw9MRhRBCiD5V1Pj/t3fncVHV+//AX2f2DQYGhk1k\nEUFx3xUN3NLU0rzVrSTLyvb126+bWbeybve2aT3qVvdWtmuLN7OuLTfMUitFRTMXFAUXQPadAWYY\nZub8/gBGCRSJZebA6/l48JiZc87MvBhHzrzn8znvkwkBAuIuoEArrrRi7aZj0KjkuHXBUMhl/AhC\nRNQW/nXshKSRYRAB/HKwsc1+Yr8ETAufgoLaIryb/hGcLqdnAxIREXUTm8OGk9U5iPTtD51Se95t\nHU4XVm9saqk/Ow5BfuffnoioL2OB1gkTBgdDq5bj5wMFcLlEAMAVAy/DENMgHC47ig1ZX3s4IRER\nUffIrDwBl+i6oOmNXxL0rk0AACAASURBVG0/heP51Zg4JBgJQ9lSn4jofFigdYJaJcfEISGosNTj\n4IkyAIBcJsfNw5IRqg/G1tPb8dPpVA+nJCIi6nru9vrtnP/sWG4lvk5tbKl//Wy21Cciag8LtE5q\nbhbyw97T7mVahRZ3jLgJBqUen2X+F0fKj3kqHhERUbfIKM+ESqZEtDHynNs0ttRPBwDctmAIdBpF\nT8UjIpIsFmidFBXii8ERfjh0shx7j5a4lwdqTbht+BLIIOCdQ2tRWFvswZRERERdp8JWicK6Ygz0\nHwCF7NxF18btp1BWXY/5k6MQG86W+kREF4IFWhe4/pJBUMgFrP3+KOpsDvfyGL8oJA++ClaHDf8+\n8B4s9TUeTElERNQ1jlZkAQDi/c89vbGqph5b9+XB5KvGpQlRPZSMiEj6WKB1gdAAPS6bHIWqGjs+\n33a8xbqJoWNxSeQMlFrLsGr7W3C4HOd4FCIiImk4c/zZuU9Q/e3OHNgdLlyaEAWlgh83iIguFCeD\nd5F5kyKx+0gxtuzLw6ShwS2mclw2YDaK6krwW8lBfHr0C1w3+CoeJE1E1Ivs2rUL999/P2JjG0eU\n4uLicMstt2DZsmVwOp0wm81YuXIlVCqVh5N2niiKyKjIhK/KB6H64Da3qaypx9bf8hDgq0biiNAe\nTkhEJG38SquLKOQy3DhnMADgg++OwuF0udfJBBluGHINBvhHILUgDT/k/uSpmERE1E0mTJiANWvW\nYM2aNXj88cfxz3/+E8nJyfj4448RGRmJ9evXezpil8ivLYTFXoNB/rHn/LLx253ZaHC4cOnkKCjk\n/KhBRNQR/KvZhQaGGzF9dD/kl9bifzuzW6xTy1VYdtGdMKp88WXWtzhQku6hlERE1BN27dqFmTNn\nAgCmT5+O1NTecdqV5umN8edor19hqcfWffkI8NXgouEcPSMi6igWaF3syqkxMBpU+GrHKRSU1bZY\nZ9L54Y4RN0IhU+C9w58g15LvoZRERNTVsrKycMcdd2DRokXYvn07rFare0pjQEAASkpK2nkEaWgu\n0AaZ2j5B9bc7s+FwujB/CkfPiIj+CB6D1sV0GgUWz4rD618cwoffHcVDyaMhO2sKSIRvOG4cci1W\nH1qDNw+8j4fG3QOj2teDiYmIqLOioqJwzz33YO7cucjNzcUNN9wAp9PpXi+K4gU9jr+/DgqFvNN5\nzGafTj9GWxqcDThedRLhvqGIDQ9vtb6syoqf9ucj2KTD5dNjO1ygdVfu7iTFzIA0c0sxMyDN3FLM\nDEg39++xQOsGY+LMGB0biH2ZpfjlQAGSRoa1WD8qaDgWDJiDjSe+w5sHP8D/jb4DKrnSQ2mJiKiz\ngoODMW/ePABAREQEAgMDcfDgQdhsNmg0GhQVFSEoKKjdx6moqOt0FrPZByUllk4/TluOVRxHvdOO\nWN+YNp9j7aajaHC4MG9iBCrKa9t4hHPrztzdRYqZAWnmlmJmQJq5pZgZkF7u8xWTnHvQDQRBwHWz\n4qBRyfGfH7NQVWtvtc3syOmYGDIW2dW5WHNkHVyiq41HIiIiKdi4cSPeeecdAEBJSQnKyspwxRVX\nICUlBQCwadMmJCYmejJilzjTXr/18Wfl1Tb8tD8fZj8NEoaF9HQ0IqJegwVaNzH5anDl1BjU1Tvw\nyeZjrdYLgoBFg69EjDEKvxYfwLcnN3sgJRERdYUZM2YgLS0NycnJuOuuu/Dkk0/igQcewJdffonk\n5GRUVlZi4cKFno7ZaRkVmZAJMgz0i2617pvUbDicIi5j50Yiok7hFMduNH10P+xML8TuI8WYPKwU\nM383lKmUKXDr8Buwcs9r+N+pzQjRmTEuZLSH0hIR0R9lMBjwxhtvtFr+3nvveSBN96hrqENO9WkM\nMEZBo9C0WFdW1Th6FuSnxWSOnhERdQq/4upGMpmAJXMGQy4TsCblGKz1jlbb+KgMuGPEjdDINViT\n8RlOVmW38UhERESedbTiOESIbbbX/2ZnNpwuEfOnREEu40cLIqLOuKC/oseOHcPFF1+MtWvXtlo3\nY8YMJCcn4/rrr8f111+PoqKiLg8pZeFBBsyZGIGyahs+Tsloc5swQwhuHnYdnC4nXtn3Jt5P/wTH\nKrJ4XBoREXmNjPLG6fqDfleglVZZ8fP+fAT7azFpaLAnohER9SrtTnGsq6vD008/jYSEhHNus3r1\nauj1+i4N1pvMnxyFtIxibPzpOIZH+SM6tHVb/aEBg3DT0EX4+sQmpBXtQ1rRPgRqTEgIG49JoePg\npzZ6IDkREVGjjIosaBUaRPq0bK//TSpHz4iIulK7f0lVKhVWr159Qe2BqW0qpRxLLhkElwh88L8M\nOF1tj4yNDR6FJyY9hAfG3ImJIWNRZbfgqxMpeGz7M/jX/nfxW/FBOFytp0kSERF1p1JrOUqtZYjz\ni4FcduY8baWVVvxyoADBJh0mDuHoGRFRV2h3BE2hUEChOP9mK1asQF5eHsaOHYsHH3wQwlknZv49\nbz8JZ3cxm33w6/Ey/JCWix2HS3DF9IHn3DYoaAQSYkegrsGKHTl78OOJHUgvy0B6WQZ81QYkRU3C\njAGTEe4b2qP5pUaKmQFp5pZiZkCauaWYmaTvXNMbv049BadLxAKOnhERdZlOd3G87777kJiYCKPR\niLvvvhspKSmYM2fOObf39pNwdqeb5w/D7vRCfPTdEQwO94XZT9vufUb6jsLIUaOQV1OA1II07C78\nFV8f3Yyvj25GtG8kJoeNx5igkdAo1N2WW4qvtxQzA9LMLcXMgDRzSzUzSV9GRRaAluc/K6m0YvvB\nQoSYdJgYz9EzIqKu0umvuxYuXIiAgAAoFAokJSXh2LHW5/yiRr56FRbNjIXd4cKalKMQRfGC79vP\nEIqrYhfgH1Mew81Dr0O8KQ6nqnPwUcZ6PLL9aaw98hlOVJ3q0GMSERG1xyW6cKw8C/5qPwRpA93L\nv9pxZvRMJjv3zBkiIuqYThVoFosFS5cuhd1uBwCkpaUhNrZ1+106Y+KQYAyLNuHQyXLsOtzxjpdK\nmQJjg0finlG34KmE5ZgXPQsGpR6pBWl4ce+/8PSuF7E5Zxss9ppuSE9ERH1NriUPtY46xJti3Ycw\nFFfUYcfBQoQG6DCBo2dERF2q3SmOhw4dwvPPP4+8vDwoFAqkpKRgxowZCA8Px6xZs5CUlIRrrrkG\narUaQ4YMOe/0RgIEQcD1lwzC42/vwic/ZGLYgAAYtMo/9FgBWn9cGj0Lc6Nm4mhFFlLz07C/5BC+\nyPoGXx3/DlP6TcTsyOnsAElERH/Y0fLG6Y1nH3/21Y5TcIkiFkyJ5ugZEVEXa7dAGzZsGNasWXPO\n9UuWLMGSJUu6NFRvZ/bTYmHiAPxnSxbW/ZiJpZcO6dTjyQQZ4k1xiDfFoaahFmmF+7Al9xdsO70D\nO/J3I7FfAmZFToOviseCEBFRxxypyAQADPJvbG5VVFGH1ENFCAvUY/xgdngmIupqbLnkIbPGhyMi\nyIDtBwtx5FR5lz2uQanH9P4XYcWkh5A86EoYlAb8mPszVux4Dl9mfYsae22XPRcREfVudqcdJypP\nor8hDD4qAwDg6+3No2c89oyIqDuwQPMQuUyGJXMHQxCAD1KOwt7g7OLHl2NKv4lYkbAM18QthFah\nxfc5W/FE6rP46kQK6ho6302TiIh6t+OVp+AQne7pjUXlddiRXoh+Zj3GcfSMiKhbsEDzoOhQX8wa\n1x/FFVZ8teNUtzyHUqZAUvhkPJXwMK6KXQCVXIXvTv2AJ1Kfw7cnv4fVYe2W5yUiIuk7UtHYmbm5\nvf7G7acgisDlU6IhO885T4mI6I9jgeZhCxOjEeCrwXe7cnC6uPs6LyrlSkzvfxH+lrAcfxp4KeSC\nHN+c/B5P7HgOKad+hM1R323PTURE0pRRngmFTIEYYzQKymqx83Ahws16jBlk9nQ0IqJeiwWah2lU\nClx/SRycLhHvf5cBl6t7z2OmkqtwccRUPJXwMBYMaOy4ufHEd1iR+hw252yD3Wnv1ucnIiJpsNhr\nkFdTgBhjFFRyJb7a0Th6toCjZ0RE3YoFmhcYEROICfFBOJFfjS378nrkOTUKDS6JmoG/TV6OS6Nn\nweFy4ousb/BE6nPYkvsLGpwNPZKDiIi809Hyxu6Ng02xKCirxa7DRQg3Gzh6RkTUzVigeYlFF8dB\np1bgs61ZyC609NjzahVazIuehacnL8ecyBmwO+1Yn7kRK1Kfx0+nd6DB5eixLERE5D2a2+sPNsWe\nOfbsIo6eERF1t3bPg0Y9w6hX4eZL4/H6hoN4Zf1+PHbDOJh8NT32/DqlDvNj5mB6/0RsztmGbae3\nY92xL/HdqR8R7hcCmUsBjVwDjUINjVwNjUINtbzxulqhhlaugbppnVquhrZpvVwm77HfgYiIuoYo\nijhangW9Uge5zQ+7Dx9DRJABY+ICPR2NiKjXY4HmRcbEmfHn6QPxny1ZeGX9ASy/bgy06p79JzKo\n9Fg4cB5mRCTi++yt2JGfhvTiY3/48ZQyBdRyNXQKLcYFj8LsqBlQyvi2IyLyZsV1Jaior8SYoBGN\nx56hcfRM4OgZEVG34ydlL3PJhP4orrRi6748vLkxHfdeORxyWc/PRPVV+eDK2Pm4MnY+TAE6nC4s\nhc1ZD5ujHjZnPeqbLhuX2VDftK7eWQ9r0+XZ21XVV+PbU5vxa8lBLB58FaKNkT3+OxER0YVpnt4Y\noozAhiPFiAz2wahYjp4REfUEFmheRhAEXDcrFqWVVhw4XoZPN2fhutlxHs0kl8mhU+qgU+r+8GPY\nHDb89/h3+ClvB17c+y9MC5+CywZcAo1C3YVJiYios+qddmzL3Q4BArIyVBDh4OgZEVEPYpMQLySX\nyXDnwmEIN+vxw6+n8f2eXE9H6jSNQoNrBi3EA2PuhFkXgC2nf8Ezu1/CkfI/Pn2SiIi63ueZX6HY\nWorxgRNx4EgdIkN8MHJggKdjERH1GSzQvJRWrcD9V42EUa/Cp5sz8VtmqacjdYmBftF4dPwDmB05\nHRX1VXjtt7ex5sh/UNdQ5+loRER93oGSdGzP34V+hlBYjg/gsWdERB7AAs2LBRg1uO+qEVAqZHhj\n46Eebb/fnZRyJS6PmYtl4+5Ff0MYdhbswd92rcK+4oOejkZE1GdV1VvwUcZ6KGQKzAm5HL8eLUd0\nqA9GxnD0jIioJ7FA83LRob64bcFQNDS48PL6/Sivtnk6Upfp79MPD427F5fHzIXVYcPbh9Zg9cEP\nUVVf7eloRER9iiiKWHvkP6hpqMXCmHk4kF4PgKNnRESewAJNAsbEmXH1jIGoqrHj5c8OwFrfe04e\nLZfJMTtyOh4d/3+IMUbjt5JDeHrXi9iRnwZRFLvlOV2iC/k1hcisOIEKWyVcoqtbnoeISCp+ykvF\n4fKjiDfFYVr4FJzIr4ZKKcOwaI6eERH1NHZxlIjZ4/ujuMKKLR5uv99dgvVB+L8xt+OXvF347/Fv\n8VHGZ9hb9BsWDb4CgdrOfUCwOxuQXZ2LE1WncLzqFE5UZcPqsLrXK2QKBGhMMGtNCNQGIFAbAHPT\nZYDWxPO2EVGvVlBbhC+yvoZeqcP18VejweFCfmkdBoT5Qibj6BkRUU/jJ0+JEAQBybNiUVLV2H7/\nk82ZuG5WXK+aeiITZEgKT8DwwHh8cnQD0ssy8I9dL2F+zBxMC58CmXBhBanFXoOTp49jX+4RnKg8\nhRxLHpyi070+UGPCiMAhMKp9UWotQ6m1DCXWchTVFbd6LAEC/NRGBGpN7qLt7AJOp9R22e9PRNTT\nGlwOvJ/+CRpcDtw0NBlGtS9OFlTDJYqIDPbxdDwioj6JBZqEyGUy3Hn5MDy7di9+/DUPwf46zBrf\n39Oxupy/xg93jrgJe4p+w2eZ/8XnmV9hb9F+XDf4KoQZQlpsK4oiiq2lOFHZODp2vOokiuvOdLyU\nCTKEG8IQ4xeFAcYoxBijYFT7tvm8tQ11TcVaWYvLUms5MitPILPyRKv76BU6+GmM7iwuiBBFESJc\njbdFEWLTMpfocl8X0fK2CyIEAEGGQJjVgQjRBSFEH4QQfTCCtIFQypVd9wITETX55sQmnK7Jx+TQ\nCRhpHgYA7oZUEcEGT0YjIuqzWKBJTHP7/b9/uAef/pCJQD8NRseaPR2rywmCgPEhozHYFIv1mRux\np+g3PJf2Ci6JmoEhprjGqYpNRVlNQ637fhq5GvGmOAwPG4QQRSiijBFQy1UX9Jx6pQ56pQ6Rvq2L\nXruzAeW28qairfys4q0MZdYKCELjaJtMkEGAAEEQztwWGi/lMjlkggABTcuatmu+dIouFNeWIbcq\nv+VrAQGBWlNjwaYLbircghCsC4JWoencC019kkt0oc5hRa29FjUNdahpqEVtQy1qmn/stVDIFPhz\n7AJ+OdCLHas4js0522DWBuDK2Pnu5dlFjQVaZAhH0IiIPIEFmgQFGDW4/88j8NxHv+LNjelYft0Y\nRIW0PSokdT4qA24amoxxwaPw6dEv8O3J7/Htye/d6/3URowNGokYv2gMMEahnyEEMkEGs9kHJSVd\nd1oClVyJEH0wQvTBXfaYbQkI1CPz9GkU1hajsK648bK2GIV1RThYegQHcaTF9n5q41mjbUFN14Ph\no/Keb76tDhtqG2qhUWiglWsgl8l75HnrnXZY7BZU2y2ottc0Xq+3oLqhBnUNdTD7+kMHA0waf5g0\nfjBp/OGjNHjltGGX6ILT5URdgxU19lo4RAecLiccorPpsum2ywm7y44a+5liq7HwqkON/UwRVttQ\nBxHnb8KjlqswN2om/OV+PfRbUk+qa6jDB4c/hSAIWDJkETQKtXtdTpEFCrmAsEC9BxMSEfVdLNAk\nKirEF7fPH4rXNhzEK+sP4PEbxsHk23tHU4YHDsFAv2h8n70NVocVMcYoDPCLgknj7+loXUomyJoK\nBn8MCRjUYp3FXuMu3Ipqi1FQW4TCumJkVGQioyKzxbY+KgP6G/oh3CcM4YZQhPv0g1kbcMHH8f1R\nNQ21OG3JR64lz/1TbG15knW1XAWtQgudQgutQgOtQtt4W9l8XdO07nfXlRooBAVqGmpRbbc0FV81\nTddrWhRj1XYL7E77+cO2PuQQSpkC/ho/mNT+CND6u/8tmos4o8r3ggtMURRhc9ajtqEOtU1FUfNP\njft206WjDnanHQ6XE07RCYfL0XTZeLsrOo0KEKBTamFQGhCsM8Og1EOv1MOg0kOv1MGg1Df+qBov\nfVU+UF3g6DNJiyiK+PToF6isr8Jl0bMRbYxwr3M4XcgtrkW/QAMU8t7TiIqISEpYoEnY6Dgzrpkx\nEJ/+mIWXPzuARxaPgVbde/9JtQotFsTM8XQMj/FRGeCjMiDWf0CL5TaHDUV1Je7iraC2EHk1hThc\nfhSHy4+6t1PJVeinD0V/nzCEG8IQ7hOGMH3IH57CVm23INeSh7LiEmQUnUSuJQ/ltooW2+gUWgzy\nHwg/tRE2Zz2sDVZYHVbUOWyorK9CQW1RuyM5HSETZPBR6hGkDYSvygc+KgN8VT7wVRngo/JxL9Mp\ntVDoRRwvOI1yWyXKbRUos1WgvOmnuK4UqGj78f3URpg0fgjQmOCv8YPT5XQXWc0FV01DLeoarC2a\n05yPUqaERqGGXJBDIcihVqqhkMkbb7svFdBq1HA1iO7b7vUyORSCAnKZHCqZ0l1k6ZuLLqUeOqW2\n2wt0koa0on3YW7wfA4yRmB05vcW6grI6OJwuRIZ4zyg8EVFf03s/zfcRs8b3R1GlFVt+zcMb/03H\nfVf1rvb71D6NQoNI3/6tjp2rbahDXk0+ci35OF2Tj9OWfGRbcnGyOtu9jUyQIVhnRrihH8J9Qt2F\nm0F5ZmqTKIqorK9CzlmjYrmWPFTZW55Q3KDUY4hpEPr79HP/BGj8zztl0CW6UO+0w+qwwuqwoa6p\ngLM6bKhz/P66DQ3OBhhU+t8VX2eu65W6Cy5CzP4+MDjanr5nc9Sjor7SXbCVWZuLt8ZlxytPIQsn\n27yvTqGFXqlDgMbkPq6xeYSq8boeeoXurHV6qC6wSO7qqbvU95RZy7Hu6JfQyNVYMuTaViPCOc3H\nn7GDIxGRx7BAkzhBEJB8cSxKK204eKIMH2/OxOJe1n6f/hi9Uoc4/4GI8x/oXtbgbEBBbRFya/Jw\n2lKA0zV5OF1TgILaIqQVnbmvv9oP4T6hcLicyLXktWjEAjQe+zY8cAj6+/TDsH4DYRRNMKp8O/y+\nkwmypmmO3jU9V6NQI1QRjNBzHHPY4HKg0laFivoKyAWFu9jSKbQ9dowdUUe5RBc+OPwpbE4bro+/\nus1zTJ7p4MgCjYjIU1ig9QJymQx3XD4Uz679FVua2u/P7oXt96nzlHIlInzDEeEb7l7mEl0otZa1\nGGk7XZOPg6WNDUkCNCYM9BuAiLNGxs5uQtIXR3WUMgXMugCYdZ07iTpRT9qUvRXHq05htHk4JoaM\nbXOb7CILBAEID+IURyIiT2GB1kto1Qr8359H4OkP92DdD5kwGzUYHdf72u9T15MJMgTpzAjSmTE2\neKR7ebXdAoUgh06p82A6Immx2Wy47LLLcNdddyEhIQHLli2D0+mE2WzGypUroVJ5pvFKdnUuvjm5\nCX5qIxYNvrLN0W6XKCKnuAZhAXqolRwJJiLyFB6s1IuYfDW4/6oRUCpleHNjOlLTCz0diSTMV+XD\n4oyog/7973/DaGw8ef0///lPJCcn4+OPP0ZkZCTWr1/vkUz1TjveP/wJXKIL18dfDf05/l8XV1hR\nb3fyBNVERB7GAq2XiQrxxV0Lh0OQCVj91WG8881h2OwOT8ciIur1jh8/jqysLEybNg0AsGvXLsyc\nORMAMH36dKSmpnok14bMr1BcV4oZ/RMx2BR7zu2ajz9jgxAiIs9igdYLjYgJwJM3jUdkiA+2HyzE\n397f4+7MRURE3eP555/H8uXL3betVqt7SmNAQABKSkp6PNOBknT8kr8L/QyhWBAz97zbZjd3cAxh\ngUZE5Ek8Bq2XCvbX4a/Xj8X6rcexKS0Xf/9wL66ZMRAzxvRjh0cioi725ZdfYtSoUejfv+0GTaJ4\nYef78/fXQaHo/PFfZrMPKq1V+Hj751DKFHhgylKE+fmf9z6F5VYAwOghodBr/9j5ETvLbJZecSjF\nzIA0c0sxMyDN3FLMDEg39++xQOvFFHIZrp0Zi/hIf7zzzRF89P0xHD5VjpvmxcPgoZ0vEVFvtHXr\nVuTm5mLr1q0oLCyESqWCTqeDzWaDRqNBUVERgoKC2n2cioq6Tmcxm31QXFyNfx94H5b6GlwVuwDa\nBt/zdlsVRRGZuRUI8tOirsaGuhpbp3N0lBQ7wkoxMyDN3FLMDEgztxQzA9LLfb5ikgVaHzByYCCe\nunkCVn+Vjn2Zpch+bzdumz8Ucf3bPkkvERF1zMsvv+y+/uqrr6Jfv37Yt28fUlJScPnll2PTpk1I\nTEzssTw/56UivSwD8aY4TA2f3O72ZdU21NociI8y9UA6IiI6Hx6D1kf4+6jxl2tH40+J0aiw1OP5\nj3/Fxu0n4XJd2LQbIiLqmHvvvRdffvklkpOTUVlZiYULF/bI856uLsCGrK+hV+qwOP7PkAnt7+pz\nimoAAJHs4EhE5HEcQetDZDIB86dEY1CEP97cmI4vfz6JjOwK3Dp/KPx91J6OR0TUK9x7773u6++9\n916PPrfD5cCrqe+hweXAjUOT4ac2XtD92MGRiMh7cAStD4rr74enbp6A0bGByMipxIp3d2N/Vqmn\nYxERUSf979QPOFmZi8mh4zHKPOyC79fcwTGCBRoRkcexQOujDFol7rliOK6bFQeb3YlX1h/Apz9k\nwuF0eToaERH9QYW1xehvDMOVsQs6dL+cIgv8fdTw1au6KRkREV0oTnHswwRBwMyx4YgNN+KN/6Zj\nU1oujuZW4o7LhyLYX+fpeERE1EG3DFuMgEA9yssuvBtkVU09KmvsGDUwsBuTERHRheIIGiEi2Acr\nbhyPi4aHIrvQgiffS8PO9EJPxyIiog4SBAFyWcfOo5bd1CAkgg1CiIi8Ags0AgCoVXLcfGk8bp0/\nBADw1leH8e43R1Bvd3o4GRERdaecIjYIISLyJpziSC0kDA3BgDBfvPHfdPxysABZeVV4eMl4GNUd\n+0aWiIikoblBSGQICzQiIm/AETRqJdhfh79ePxazx/dHYXkd/vLKT/hq+0k4XWwgQkTU22QXWmDQ\nKnm6FSIiL8ECjdqkkMtw7cxYPHjNKPj5qPHFzyfxzJpfUVBW6+loRETURWptDSitsiEy2ABBEDwd\nh4iIwAKN2jE02oTX/jIdCUODcbKgGk+9l4bNe3LhEkVPRyMiok7KaW4QwumNRERegwUatcugU+HW\n+UNx18JhUCnl+HhzJl5a9xvKq22ejkZERJ2QXcgGIURE3oYFGl2wcYOD8PTSCRgZE4DDpyrw+Du7\nseNQAUSOphERSVJOMQs0IiJvwwKNOsRoUOO+q0bgxrmD4RJFvP31Efzri0OorrN7OhoREXVQdqEF\nGpUcZn+tp6MQEVETttmnDhMEAUkjwxAf6Y93vj6MvcdKkHm6EkvmDsboWLOn4xER0QWotztRWFaH\n2P5+kLFBCBGR17igEbRjx47h4osvxtq1a1ut27FjB6666ipcc801eP3117s8IHkvs58Wy5LH4Orp\nA1FX78Crnx/Eu98egbXe4eloRETUjtySGogAIoINno5CRERnabdAq6urw9NPP42EhIQ21//973/H\nq6++ik8++QTbt29HVlZWl4ck7yWTCZgzMQJP3DgeEUEG/HKgACve3Y2jORWejkZEROfBBiFERN6p\n3QJNpVJh9erVCAoKarUuNzcXRqMRoaGhkMlkmDp1KlJTU7slKHm3cLMBjy0Zh8smR6Gs2oYXPt6H\nT3/IRIPD6eloRETUhuyipgKNLfaJiLxKuwWaQqGARqNpc11JSQlMJpP7tslkQklJSdelI0lRyGW4\nImkAHl08FkH+ggjPtQAAHuhJREFUWmxKy8VT7+9xf0tLRETeI6fIAqVChtAAnaejEBHRWXq8SYi/\nvw4KhbzTj2M2S/Mbv76Q22z2waj4ELz/zWF8s/0k/v7hHlw7exD+PCMWcnnPNQ7tC6+1t5BiZkCa\nuaWYmbxPg8OFvJJaRAT7QC5jQ2ciIm/SqQItKCgIpaWl7ttFRUVtToU8W0VFXWeeEkDjB5SSEumN\nyvS13FcmRmNwuBHvfnsEH32XgR/TcnDVtBiMGhgIoZs7hvW119qTpJgZkGZuqWYm75NfWgunS+T0\nRiIiL9Spr83Cw8NRU1OD06dPw+FwYMuWLZgyZUpXZaNeYGi0CX9bOgFTR4WhsLwOr35+EM999Cuy\nTld5OhoRUZ/VfPwZOzgSEXmfdkfQDh06hOeffx55eXlQKBRISUnBjBkzEB4ejlmzZuHJJ5/Egw8+\nCACYN28eoqOjuz00SYteo8SSOYMxa1x/fL7tOPZlluKZtXsxOjYQV02LQWiA3tMRiYj6FHeDEHZw\nJCLyOu0WaMOGDcOaNWvOuX78+PFYt25dl4ai3iksUI97rxyBzNOV+GxLY6G2P6sMiSNDcflF0fAz\nqD0dkYioT8gptEAuExBu5hdkRETepsebhBDFhvvhkcVj8FtmKdZvO45tv+UjNb0Qs8f3x9yJkdCq\n+bYkIuouLpeI3OIahAbooeyCpl1ERNS1+EmYPEIQBIyOM2PEwABsP1iIL38+ga93ZGPrvnzMnxyF\naaP7QalgZzEioq5WUF4Hu8OFyBAef0ZE5I34CZg8Si6TIWlkGJ69PQFXJA2A0+XCJz9k4q+rd2Ln\n4UK4RNHTEYmIepWcQh5/RkTkzVigkVdQK+W4bHIUnrs9AbPG9UeFpR5vbTyMp9/fg/RT5Z6OR0TU\na5zp4MgCjYjIG7FAI6/io1Nh0cWxeOa2SZg0NBjZRRa8+OlveHHdb8gpktb5n4iIvFFOkQUCgP5B\nnOJIROSNeAwaeSWznxa3zR+KS8ZHYP2240g/WY70k+WYNDQYVyQOQKCf1tMRiYgkRxRFZBfVINik\nY0MmIiIvxb/O5NUiQ3zw4DWjkH6qHJ9tycLO9CLsySjGtNH9cNnkKPjqVJ6OSEQkGSVVNljrHRg+\nwOTpKEREdA4s0EgShkaZEH/jeOw+XIQNP53A5j2n8cuBAsyZEIHZE/pDo+JbmYioPe4GISE8/oyI\nyFvxUy1JhkwQMGloCMYNDsK23/KxcftJfPnLSfz462nMnxKNqaPCoJDzsEoionNpbhDCDo5ERN6L\nn2ZJchRyGWaODcdztyfg8ouiUe9w4aPvj+Gvq3di1+EituYnIjoHdnAkIvJ+LNBIsrRqBS6/KBrP\n356AmWPDUV5djzc3puNv76fh16PFEFmoERG5iaKI7EILAnw1MGiVno5DRETnwAKNJM9Xr8J1s+Lw\nj6bW/LlFNVjxVipWffobThZUezoeEZFXqKyxw1LXwOPPiIi8HAs06jWCmlrzr7hpPMYMDsKR7Ao8\n/cEe/OvLQygqr/N0PCIijzpz/BnPf0ZE5M3YJIR6nYhgHzx1awJ+2pOD9VuPY09GMX49WoKkUWFY\nMCUKfga1pyMSEfW45g6OPP6MiMi7sUCjXis+0h+P3TAWe4+W4POfTmDrvjzsOFSAWeP6Y+7ESOg0\nfPsTUdewWq1Yvnw5ysrKUF9fj7vuuguDBw/GsmXL4HQ6YTabsXLlSqhUnjt3o3sEjVMciYi8Gj+h\nUq8mCALGDQ7C6LhA/HygAP/95SS+Sc3G1n15mD2+P0bHmdEvUA9BEDwdlYgkbMuWLRg2bBhuvfVW\n5OXl4eabb8aYMWOQnJyMuXPn4qWXXsL69euRnJzssYw5RRYY9SrOIiAi8nIs0KhPkMtkmDaqHxKG\nhmDznlx8uzMHX/x8El/8fBImXzWGRQdg+IAADInyh1bN/xZE1DHz5s1zXy8oKEBwcDB27dqFp556\nCgAwffp0vPvuux4r0Cx1dpRV12P4gACPPD8REV04fhKlPkWtlOPShChMG90P+7NKcfBEOQ6dKMNP\n+/Px0/58yGUCYsONGB7TWLBxdI2IOuLaa69FYWEh3njjDdx0003uKY0BAQEoKSnxWK6cohoAQGQI\nG4QQEXk7FmjUJ+k1SkweForJw0Lhcok4UVCNg8fLcPBEGTJyKpGRU4nPthzn6BoRdcinn36KI0eO\n4KGHHmpxLsYLPS+jv78OCoW80znM5pbHmZUdLAQADI8NarXOm3hztnORYmZAmrmlmBmQZm4pZgak\nm/v3+GmT+jyZTMDAfkYM7GfEn5IGoLrWjkMny3DgeBnST5ZzdI2I2nXo0CEEBAQgNDQU8fHxcDqd\n0Ov1sNls0Gg0KCoqQlBQULuPU1HR+VOCmM0+KCmxtFh2+EQpAMBPq2i1zlu0ldvbSTEzIM3cUswM\nSDO3FDMD0st9vmKSBRrR7/jqVR0aXRsVG4gRMQGQsVgj6rP27NmDvLw8/PWvf0VpaSnq6uqQmJiI\nlJQUXH755di0aRMSExM9li+70AKdWoFAo8ZjGYiI6MKwQCM6j9+PrlXV2nHoRGOxdvbo2sB+Riye\nHcfzCxH1Uddeey3++te/Ijk5GTabDU888QSGDRuGhx9+GOvWrUNYWBgWLlzokWzWegeKKqyIj/Tn\nqD8RkQSwQCPqAKNehSnDQzFl+JnRtZTdOdh7tARPvZ+GaaP74YqkAdBrlJ6OSkQ9SKPR4MUXX2y1\n/L333vNAmpZyixsbhEQEs0EIEZEUsEAj+oPco2t/Go70k+X46Ptj2PJrHtKOFOOqaTG4aEQopz0S\nkcdlFzadoJoj/EREkiDzdACi3mBotAl/WzoBf54egwaHC+//LwP/+HAPThZUezoaEfVx2UVNBVoI\nCzQiIilggUbURRRyGeZOjMQzt03CxCHBOFlgwd8/2IP3/5cBS53d0/GIqI/KKbJApZQh2F/n6ShE\nRHQBWKARdTF/HzVuXzAUyxaNRligHj/tz8ejb+3Ell9Pw+W6sHMhERF1BXuDE/mldYgI8oFMxinX\nRERSwAKNqJsMjvTHipvG49qZsXCJItZsOoa/fZCGrLwqT0cjoj7idEktXKLI48+IiCSEBRpRN1LI\nZZg9vj+euXUSJg8LQU5RDZ5ZsxfvfH0YVbWc9khE3Sun6fgzdnAkIpIOFmhEPcBoUOOWy4bgkcVj\nEBFkwPZDhXj0rVR8n5YLp8vl6XhE1EuxQQgRkfSwQCPqQbHhfnjixvFYPDsOAgR88kMmnnwvDUdz\nKjwdjYh6oexCCxRyAWGBek9HISLyelu3/nBB273yyovIz8/rthw8DxpRD5PJBMwYE45xg4OwYdtx\n/Ly/AM9/vA8T4oOQOKY/9EoBwf46aNX870lEf5zD6cLpklr0CzRAIef3sURE51NQkI/Nm1MwbdrM\ndre9//4HuzULPwESeYivToUb58YjaWQ/rN10FLuPFGP3kWL3eqNBhRB/HYJNOoQ0/wToEGjU8MMW\nEbWroKwODqcLkSE8/oyIqD0vvfQ8jhxJR2LieMyePRcFBfl4+eV/4dln/4aSkmJYrVbcfPNtmDIl\nEffccxv+3/9bhi1bfkBtbQ1ycrKRl3ca9933IBISpnQ6Cws0Ig8bEOaLx5aMw9HsClTXO5GVXYHC\nijoUldfhWG4ljuZWttheLhMQ6KdFiL8WIQFNBZx/Y/Fm1KsgCGylTUSN0xsBsIMjEUnOf37MQlpG\ncfsbnkUuF+B0nvt0RuMHB+HqGQPPuX7RouuxYcN/EB0dg5ycU/jXv95GRUU5JkyYhLlzL0Ne3mk8\n/vhyTJmS2OJ+xcVFWLXqn9i5cwf++9/PWaAR9RYyQUB8lAlmsw9KBlncy+0NThRXWlFYVoeiijoU\nltWhsOlyf3kd9h8va/E4GpUcwSYdzH5aBBo1CDRqEODbdGnUQKPif3mivuJMB0cWaEREHREfPxQA\n4OPjiyNH0rFx4wYIggzV1a1PlTRixCgAQFBQEGpqarrk+flpjciLqZRyhJsNCDe3nqJUY21AYXld\nq+Itr6TW/c357xm0SgQ0FW6NP9oWBRyPeyPqPbKLLBAEIDyIUxyJSFqunjHwvKNdbTGbfVBS0vbn\nn45SKpUAgO+//w7V1dV4/fW3UV1djVtuub7VtnK53H1dFM89gtcR/DRGJFEGrRID+xkxsJ+xxXKX\nKKKqxo6yKhtKq62Nl1U29+X5Cji9RtFUwGndRZu56XqgH0fgiKTCJYrIKa5BWIAeaqW8/TsQEfVx\nMpkMTqezxbLKykqEhoZBJpNh27Yf0dDQ0CNZ+GmLqJeRCQL8fdTw91FjIIyt1rtEEZZaO0qrzxRt\nZwq4xumUOUVtD9EbtEqY/TQIMGphNmoQ6KdFTIQ/lBARaNRAqeAHQSJvUFxhRb3dyRNUExFdoMjI\naBw9moHQ0DD4+fkBAKZNm4Hly/8fDh8+hEsvXYCgoCC8997qbs/CAo2oj5EJAowGNYwGNWLCWhdw\noijCUtfQVLhZ3QVcaaUVJVU25BbX4GRB2yNwRoOqccTN78wUSrNRgwA/LfwNaigV7D5J1BPYIISI\nqGP8/f2xYcM3LZaFhobhgw8+dd+ePXsuAOCmm24FAAwYcGYa5oABA/Haa291SRYWaETUgiAI8NWr\n4KtXYUCYb6v1zVMoS6usKK20oc7hQnZ+FUorG4u5E/nVyMprfRAtAPjqVQjwVcPko4G/rxoBvhqY\nfDUw+ahh8tXAaFBBxi6URJ2W3dQgJDKEBRoRkdSwQCOiDjl7CmVseOuDcp0uFyqq61HSNOrWPAJX\nYbGhvLr+vCNwclnjYzcXbI0/jQWdyVeNQKMWOg3/bBG1p7mDY/8gFmhERFLDTzpE1KXkMhkC/bQI\n9NMCkf6t1ruaplCWV9uafupRbrGhrLoeFdU2lFXbkHm6CiJaj8IJAjCovx/GxwdjbJwZvnpVT/xK\nRJIiiiKyCy0I8uMXGkREUsS/3ETUo2SCAKNeBaNehejQ1lMoAcDhdKGypr6xeKu2odxSj7JqG3IK\nLcjIqURGTiXWbjqKwRH+GB8fhDFxZvjqWKwRAUBJhRW1Ngfio0yejkJERH8ACzQi8joKuayp1b+2\n1bryahv2ZBQjLaMYR7IrcCS7AmtTjiE+0g/jBjcWaz4s1qgPO950DGgkOzgSEUkSCzQikhSTrwaz\nJ0Rg9oQIlFZZsSejBGkZxUg/VYH0UxVYk3IM8VH+GN9UrBm0Sk9HJupRx/MqAbCDIxGRVLFAIyLJ\nCjRqMWdiBOZMjEBppRV7jpYgLaMI6SfLkX6yHGtSjrYo1vQaFmvU+x0/3TiCFsECjYioy1111Xx8\n+OE66HS6bnsOFmhE1CsE+p0p1koqrdiTUYzdGcU4dKIch06U48PvjmJotAnjBgVhdFwgzJ4OTNRN\nTuRVwd9HzSY6REQSxQKNiHods58WcydFYu6kSBQ3FWtpR4px4HgZDhwvg/AtoFbJIZcJjT9yGRRy\nAXJZ0+Xvbivkst9t17hMIZPBV6+EyVfTeE43Y+M53RRynpCbPKOqprGxzqiBgZ6OQkQkKTfffB2e\neeZFhISEoLCwAI888iDM5iBYrVbYbDY88MBDGDJkWI9kuaAC7ZlnnsH+/fshCAIeffRRjBgxwr1u\nxowZCAkJgVwuBwCsWrUKwcHB3ZOWiKiDgvy0mDcpEvMmRaKoog57mkbVnKIIW70DDqcIp8sFh1NE\nvb2h6bYIh9MFp0vs8PMJAHwNKvdJuAN81e4CLsBXgwCjBnqNAgJPyE3dILuoBgAQwQYhRCRhG7K+\nxr7igx26j1wmnHe/PTpoOK4YeNk51yclTcf27T/hyiuvxs8/b0NS0nTExMQiKWka9u5Nw0cffYB/\n/GNlhzL9Ue0WaLt370Z2djbWrVuH48eP49FHH8W6detabLN69Wro9fpuC0lE1BWC/XW4NCEKlyZE\ntTrBdltEsbFYczpFOJqKOKfTBYer8bLB4UJ1rR1l1Y3ncWs+t1tZtQ3ZhRacyK9u83FVSlmrAs5H\nq4RKKYdKKYdSIYNaIXPfVillUCnk0OrVaHC4oJALLPCoTc0nqGaDECKijklKmo7XXnsZV155NX75\nZRvuuecBfPrpGnzyyRo0NDRAo9H0WJZ2C7TU1FRcfPHFAICYmBhUVVWhpqYGBgO/nSOi3k0QhKYp\njoAa8g7d1yWK7uKtvLoeZVW2pus297KCsro/mAtQKeVQK2RQKpoKOPdtGWSyximZsuYpnE3XZTIB\nirOuN66Tua+fvb3RoMLE+GAWghKT3VyghbBAIyLpumLgZecd7WrLhXzxej4DBsSgrKwERUWFsFgs\n+PnnrQgMDMLjjz+NjIzDeO21l//wY3dUuwVaaWkphg4d6r5tMplQUlLSokBbsWIF8vLyMHbsWDz4\n4IPcoRNRnycTBPgZ1PAzqBET1vY29XYnyi02lFXZUGtzwO5wwt7gOnPZ4ITdceYSgoCa2nrUO1xo\ncG/nhKWuAXaHDQ0NLnR8Uua5xYQZYfZrfS468l6WWjsCjRr4+6g9HYWISHISEi7CW2/9C4mJU1FZ\nWYGYmFgAwLZtW+BwOHosR4ebhIhiy93/fffdh8TERBiNRtx9991ISUnBnDlzznl/f38dFIqOfRPd\nFrNZmt8OMnfPkWJmQJq5pZgZ8I7c4V34WKJ45tg5Z9OxdE6XCy6X6D7WzukU4XKdOc6u+XrzcXiu\npumbBp0KQwYEdGE66gm3LRgKo58Ogsvl6ShERJIzdep03HHHzXj//U9gs1nx97+vwJYtm3HllVdj\n8+ZN+OabjT2So90CLSgoCKWlpe7bxcXFMJvPNKheuHCh+3pSUhKOHTt23gKtouKPTek5W2eHMD2F\nuXuOFDMD0swtxcyANHN3JrOs6UcpAyAT0NjO5PzdJrvi9fGGIrgvMflqYA7QS+69TUTkDeLjh2Lb\ntl3u2x99tN59/aKLpgIALr10QbfnaLcX9JQpU5CSkgIASE9PR1BQkHt6o8ViwdKlS2G32wEAaWlp\niI2N7ca4REREREREvVe7I2hjxozB0KFDce2110IQBKxYsQIbNmyAj48PZs2ahaSkJFxzzTVQq9UY\nMmTIeUfPiIiIiIiI6Nwu6Bi0v/zlLy1uDx482H19yZIlWLJkSdemIiIiIiIi6oPaneJIRERERERE\nPYMFGhERERERkZfocJt9IiIiau2FF17A3r174XA4cPvtt2P48OFYtmwZnE4nzGYzVq5cCZVK5emY\nRETk5VigERERddLOnTuRmZmJdevWoaKiAn/605+QkJCA5ORkzJ07Fy+99BLWr1+P5ORkT0clIiIv\nxymOREREnTR+/Hi88sorAABfX19YrVbs2rULM2fOBABMnz4dqampnoxIREQSwRE0IiKiTpLL5dDp\ndACA9evXIykpCb/88ot7SmNAQABKSkrafRx/fx0UCnmn80j1BOFSzC3FzIA0c0sxMyDN3FLMDEg3\n9++xQCMiIuoimzdvxvr16/Huu+9i9uzZ7uWiKF7Q/Ssq6jqdwWz2QUmJpdOP09OkmFuKmQFp5pZi\nZkCauaWYGZBe7vMVkz1eoHVVZSvVCpm5e44UMwPSzC3FzIA0c0sxc1/x888/44033sDbb78NHx8f\n6HQ62Gw2aDQaFBUVISgoqN3H4D5SermlmBmQZm4pZgakmVuKmQHp5v49HoNGRETUSRaLBS+88ALe\nfPNN+Pn5AQAmT56MlJQUAMCmTZuQmJjoyYhERCQRgnih8y6IiIioTevWrcOrr76K6Oho97LnnnsO\njz32GOrr6xEWFoZnn30WSqXSgymJiEgKWKARERERERF5CU5xJCIiIiIi8hIs0IiIiIiIiLwECzQi\nIiIiIiIv4fXnQXvmmWewf/9+CIKARx99FCNGjHCv27FjB1566SXI5XIkJSXh7rvv9mDSll544QXs\n3bsXDocDt99+e4vz4cyYMQMhISGQyxtPRrpq1SoEBwd7KioAYNeuXbj//vsRGxsLAIiLi8Pjjz/u\nXu+tr/Vnn32GjRs3um8fOnQI+/btc98eOnQoxowZ4779/vvvu1/3nnbs2DHcdddduPHGG7F48WIU\nFBRg2bJlcDqdMJvNWLlypfukts3O9/73ZO5HHnkEDocDCoUCK1euhNlsdm/f3nvJE5mXL1+O9PR0\nd3e9pUuXYtq0aS3u442v9X333YeKigoAQGVlJUaNGoWnn37avf2GDRvwyiuvICIiAkBj18A777yz\nx3OTZ3D/2DO4f+wZ3Ed6LjP3kV5I9GK7du0Sb7vtNlEURTErK0u8+uqrW6yfO3eumJ+fLzqdTnHR\nokViZmamJ2K2kpqaKt5yyy2iKIpieXm5OHXq1Bbrp0+fLtbU1Hgg2bnt3LlTvPfee8+53ltf67Pt\n2rVLfPLJJ1ssmzBhgofStFRbWysuXrxYfOyxx8Q1a9aIoiiKy5cvF7/99ltRFEXxxRdfFD/66KMW\n92nv/d8T2sq9bNky8ZtvvhFFURTXrl0rPv/88y3u0957qbu1lfnhhx8Wf/zxx3Pex1tf67MtX75c\n3L9/f4tln3/+ufjcc8/1VETyItw/9hzuH7sf95E9h/tIafDqKY6pqam4+OKLAQAxMTGoqqpCTU0N\nACA3NxdGoxGhoaGQyWSYOnUqUlNTPRnXbfz48XjllVcAAL6+vrBarXA6nR5O9cd582t9ttdffx13\n3XWXp2O0SaVSYfXq1S1OVLtr1y7MnDkTADB9+vRWr+n53v89pa3cK1aswCWXXAIA8Pf3R2VlZY9m\nak9bmdvjra91sxMnTsBisXjkG0vyTtw/egdvfq3P5s37R4D7yJ7EfaQ0eHWBVlpaCn9/f/dtk8mE\nkpISAEBJSQlMJlOb6zxNLpdDp9MBANavX4+kpKRW0wZWrFiBRYsWYdWqVRC95EwHWVlZuOOOO7Bo\n0SJs377dvdybX+tmBw4cQGhoaItpBABgt9vx4IMP4tprr8V7773noXSAQqGARqNpscxqtbqnawQE\nBLR6Tc/3/u8pbeXW6XSQy+VwOp34+OOPMX/+/Fb3O9d7qSe0lRkA1q5dixtuuAEPPPAAysvLW6zz\n1te62YcffojFixe3uW737t1YunQplixZgsOHD3dnRPIi3D/2LO4fuxf3kT2H+0hp8Ppj0M7mLX+o\nL9TmzZuxfv16vPvuuy2W33fffUhMTITRaMTdd9+NlJQUzJkzx0MpG0VFReGee+7B3LlzkZubixtu\nuAGbNm1qNd/bW61fvx5/+tOfWi1ftmwZFixYAEEQsHjxYowbNw7Dhw/3QMLzu5D3tje9/51OJ5Yt\nW4ZJkyYhISGhxTpvfC9dfvnl8PPzQ3x8PN566y289tpreOKJJ865vTe91na7HXv37sWTTz7Zat3I\nkSNhMpkwbdo07Nu3Dw8//DC++uqrng9JHudN79kLwf1jz5H6/hHgPrK7cR/pfbx6BC0oKAilpaXu\n28XFxe5vgH6/rqioqEPDtd3t559/xhtvvIHVq1fDx8enxbqFCxciICAACoUCSUlJOHbsmIdSnhEc\nHIx58+ZBEAREREQgMDAQRUVFALz/tQYap0KMHj261fJFixZBr9dDp9Nh0qRJXvFaN9PpdLDZbADa\nfk3P9/73tEceeQSRkZG45557Wq0733vJUxISEhAfHw+gsQnB798H3vxap6WlnXPaRkxMjPtA7tGj\nR6O8vFzS08XownH/2HO4f/QM7iN7DveR3serC7QpU6YgJSUFAJCeno6goCAYDAYAQHh4OGpqanD6\n9Gk4HA5s2bIFU6ZM8WRcN4vFghdeeAFvvvmmuyPO2euWLl0Ku90OoPGN1dzJx5M2btyId955B0Dj\nlI2ysjJ35yxvfq2Bxj/cer2+1bdPJ06cwIMPPghRFOFwOPDrr796xWvdbPLkye7396ZNm5CYmNhi\n/fne/560ceNGKJVK3Hfffedcf673kqfce++9yM3NBdD4YeX37wNvfa0B4ODBgxg8eHCb61avXo2v\nv/4aQGN3K5PJ5NEubNRzuH/sOdw/egb3kT2H+0jvI4jeNE7ZhlWrVmHPnj0QBAErVqzA4cOH4ePj\ng1mzZiEtLQ2rVq0CAMyePRtLly71cNpG69atw6uvvoro6Gj3sokTJ2LQoEGYNWsWPvjgA3z55ZdQ\nq9UYMmQIHn/8cQiC4MHEQE1NDf7yl7+guroaDQ0NuOeee1BWVub1rzXQ2Dr45Zdfxttvvw0AeOut\ntzB+/HiMHj0aK1euxM6dOyGTyTBjxgyPtVc9dOgQnn/+eeTl5UGhUCA4OBirVq3C8uXLUV9fj7Cw\nMDz77LNQKpV44IEH8Oyzz0Kj0bR6/5/rj1BP5i4rK4NarXb/cY6JicGTTz7pzu1wOFq9l6ZOnerR\nzIsXL8Zbb70FrVYLnU6HZ599FgEBAV7/Wr/66qt49dVXMXbsWMybN8+97Z133ol///vfKCwsxEMP\nPeT+kOWp1sfkGdw/9gzuH3smJ/eRnsvMfaT38foCjYiIiIiIqK/w6imOREREREREfQkLNCIiIiIi\nIi/BAo2IiIiIiMhLsEAjIiIiIiLyEizQiIiIiIiIvAQLNCIiIiIiIi/BAo2IiIiIiMhLsEAjIiIi\nIiLyEv8f9sUwV2nlQuEAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<matplotlib.figure.Figure at 0x7f7070a95ac8>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "metadata": {
        "id": "GPQH0NVwQAO3",
        "colab_type": "code",
        "outputId": "5bad94de-2bb3-4e50-f9a3-6a5a521e1eb7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        }
      },
      "cell_type": "code",
      "source": [
        "# Test performance\n",
        "trainer.run_test_loop()\n",
        "print(\"Test loss: {0:.2f}\".format(trainer.train_state['test_loss']))\n",
        "print(\"Test Accuracy: {0:.1f}%\".format(trainer.train_state['test_acc']))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test loss: 1.32\n",
            "Test Accuracy: 69.8%\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "metadata": {
        "id": "w6WRq-O3d1ba",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "# TODO"
      ]
    },
    {
      "metadata": {
        "id": "oEcbaRswd1d0",
        "colab_type": "text"
      },
      "cell_type": "markdown",
      "source": [
        "* image classification example\n",
        "* segmentation\n",
        "* deep CNN architectures\n",
        "* small 3X3 filters\n",
        "* details on padding and stride (control receptive field, make every pixel the center of the filter, etc.)\n",
        "* network-in-network (1x1 conv)\n",
        "* residual connections / residual block\n",
        "* interpretability (which n-grams fire)"
      ]
    }
  ]
}
